-- Logs begin at Thu 2022-08-18 11:21:28 UTC, end at Thu 2022-08-18 12:41:32 UTC. -- Aug 18 12:00:16.025711 dv-cinder-836027-7-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 18 12:00:16.033016 dv-cinder-836027-7-iscsi devstack@keystone.service[81850]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 18 12:00:16.035021 dv-cinder-836027-7-iscsi devstack@keystone.service[81850]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Aug 18 12:00:16 2022] *** Aug 18 12:00:16.035021 dv-cinder-836027-7-iscsi devstack@keystone.service[81850]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 18 12:00:16.035021 dv-cinder-836027-7-iscsi devstack@keystone.service[81850]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 18 12:00:16.035021 dv-cinder-836027-7-iscsi devstack@keystone.service[81850]: nodename: dv-cinder-836027-7-iscsi Aug 18 12:00:16.035021 dv-cinder-836027-7-iscsi devstack@keystone.service[81850]: machine: x86_64 Aug 18 12:00:16.035021 dv-cinder-836027-7-iscsi devstack@keystone.service[81850]: clock source: unix Aug 18 12:00:16.035021 dv-cinder-836027-7-iscsi devstack@keystone.service[81850]: pcre jit disabled Aug 18 12:00:16.035021 dv-cinder-836027-7-iscsi devstack@keystone.service[81850]: detected number of CPU cores: 4 Aug 18 12:00:16.035021 dv-cinder-836027-7-iscsi devstack@keystone.service[81850]: current working directory: / Aug 18 12:00:16.035021 dv-cinder-836027-7-iscsi devstack@keystone.service[81850]: detected binary path: /usr/bin/uwsgi-core Aug 18 12:00:16.035713 dv-cinder-836027-7-iscsi devstack@keystone.service[81850]: your processes number limit is 47680 Aug 18 12:00:16.035713 dv-cinder-836027-7-iscsi devstack@keystone.service[81850]: your memory page size is 4096 bytes Aug 18 12:00:16.035713 dv-cinder-836027-7-iscsi devstack@keystone.service[81850]: detected max file descriptor number: 2048 Aug 18 12:00:16.035713 dv-cinder-836027-7-iscsi devstack@keystone.service[81850]: lock engine: pthread robust mutexes Aug 18 12:00:16.035713 dv-cinder-836027-7-iscsi devstack@keystone.service[81850]: thunder lock: enabled Aug 18 12:00:16.035713 dv-cinder-836027-7-iscsi devstack@keystone.service[81850]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 18 12:00:16.036005 dv-cinder-836027-7-iscsi devstack@keystone.service[81850]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 18 12:00:16.075072 dv-cinder-836027-7-iscsi devstack@keystone.service[81850]: Python main interpreter initialized at 0x559ab97d19c0 Aug 18 12:00:16.075072 dv-cinder-836027-7-iscsi devstack@keystone.service[81850]: python threads support enabled Aug 18 12:00:16.075072 dv-cinder-836027-7-iscsi devstack@keystone.service[81850]: your server socket listen backlog is limited to 100 connections Aug 18 12:00:16.075072 dv-cinder-836027-7-iscsi devstack@keystone.service[81850]: your mercy for graceful operations on workers is 90 seconds Aug 18 12:00:16.075490 dv-cinder-836027-7-iscsi devstack@keystone.service[81850]: mapped 403077 bytes (393 KB) for 2 cores Aug 18 12:00:16.075570 dv-cinder-836027-7-iscsi devstack@keystone.service[81850]: *** Operational MODE: preforking *** Aug 18 12:00:16.075827 dv-cinder-836027-7-iscsi devstack@keystone.service[81850]: *** uWSGI is running in multiple interpreter mode *** Aug 18 12:00:16.075827 dv-cinder-836027-7-iscsi devstack@keystone.service[81850]: spawned uWSGI master process (pid: 81850) Aug 18 12:00:16.076030 dv-cinder-836027-7-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 18 12:00:16.079451 dv-cinder-836027-7-iscsi devstack@keystone.service[81850]: spawned uWSGI worker 1 (pid: 81862, cores: 1) Aug 18 12:00:16.079451 dv-cinder-836027-7-iscsi devstack@keystone.service[81850]: spawned uWSGI worker 2 (pid: 81863, cores: 1) Aug 18 12:00:16.079451 dv-cinder-836027-7-iscsi devstack@keystone.service[81850]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 18 12:00:20.415800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 18 12:00:20.415800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: from cryptography.utils import int_from_bytes Aug 18 12:00:20.416598 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 18 12:00:20.416598 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: from cryptography.utils import int_from_bytes Aug 18 12:00:20.416708 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 18 12:00:20.416708 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: from cryptography.utils import int_from_bytes Aug 18 12:00:20.417113 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 18 12:00:20.417113 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: from cryptography.utils import int_from_bytes Aug 18 12:00:20.466904 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 18 12:00:20.467118 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 18 12:00:20.467235 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 18 12:00:20.467337 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] command line args: [] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 18 12:00:20.467453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 18 12:00:20.467565 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 18 12:00:20.467661 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] command line args: [] {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 18 12:00:20.467736 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ================================================================================ {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 18 12:00:20.467913 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 18 12:00:20.467999 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] admin_token = **** {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.468105 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ================================================================================ {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 18 12:00:20.468331 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] config_dir = [] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.468423 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] admin_token = **** {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.468530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.468617 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] config_dir = [] {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.468717 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] config_source = [] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.468803 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.468899 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] debug = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.468990 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] config_source = [] {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.469106 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.469451 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] debug = True {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.469451 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.469689 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.469789 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.469904 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.470008 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.470112 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.470219 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.470329 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.470434 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.470539 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] list_limit = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.470643 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] list_limit = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.470746 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] log_config_append = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.470850 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] log_config_append = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.470964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.471112 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.471216 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] log_dir = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.471318 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] log_dir = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.471462 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] log_file = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.471552 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] log_file = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.471683 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.471767 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.471906 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.472184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.472285 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.472371 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.472498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.472498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.472693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.472770 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.472896 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.472896 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.473091 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.473175 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.473249 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.473366 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.473455 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.473528 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.473646 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.473729 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.473809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.473926 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.474013 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.474087 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.474209 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.474302 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.474382 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.474454 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.474582 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.474667 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.474872 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.98/identity {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.475099 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] publish_errors = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.475190 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.98/identity {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.475257 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] publish_errors = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.475323 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.475522 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.475687 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.475687 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.475806 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.475954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.476040 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.476193 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.476285 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.476432 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] standard_threads = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.476519 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.476666 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.476751 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] standard_threads = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.476901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.476985 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.477083 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.477242 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.477413 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] use_journal = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.477592 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.477729 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] use_json = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.477860 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] use_journal = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.477984 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] use_stderr = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.478118 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] use_json = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.478250 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] use_syslog = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.478394 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] use_stderr = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.478533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.478659 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] use_syslog = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.478659 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 18 12:00:20.478904 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.478904 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.479056 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.479056 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.479270 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.479270 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.479404 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.479404 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.479708 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.479708 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.479847 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.479906 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.479984 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.480062 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.480135 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.480217 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.480296 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.480368 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.480439 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.480605 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.480697 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.480845 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.480930 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.481098 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.481201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.481340 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.481429 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.481594 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.481683 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.481847 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.481966 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.482086 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.482179 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.482330 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.482409 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.482591 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.482675 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.482841 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.482928 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.483081 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.483161 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.483324 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.483401 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.483619 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.483719 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.483873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.483964 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.484125 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.484205 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.484359 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.484530 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.484607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.484679 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.484816 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.484896 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.485056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.485152 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.485308 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.485387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.485560 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.485695 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.485795 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.485883 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.486039 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.486128 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.486275 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.486354 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.486523 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.486610 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.486763 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.486849 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.487007 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.487085 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.487240 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.487329 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.487503 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.487637 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.491553 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.491683 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.491791 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.491901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.492007 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.492112 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.492212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.492351 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.492467 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.492568 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.492667 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.492766 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.492766 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.492956 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.492956 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.493163 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.493241 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.493314 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.493432 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.493509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.493639 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.493769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.493848 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.493978 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.494051 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.494125 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.494243 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.494319 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.494445 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.494534 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.494609 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.494728 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.494805 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.494931 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.495010 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.495083 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.495206 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.495300 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.495375 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.495520 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.495603 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.495723 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.495800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.495918 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.495995 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.496077 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.496152 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.496271 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.496367 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.496443 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.496527 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.496603 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.496671 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.496790 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.496868 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.496985 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.497061 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.497122 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.497194 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.497261 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.497365 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.497481 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.497559 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.497632 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.497758 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.497837 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.497905 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.497977 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.498049 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.498121 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.498268 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.498345 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.498417 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.498489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.498568 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.498642 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.498728 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.498804 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.498876 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.498948 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.499067 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.499067 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.499215 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.499301 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.499376 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.499482 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.499557 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.499629 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.499701 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.499786 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.499862 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.499935 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.500007 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.500080 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.500153 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.500233 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.500307 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.500380 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.500452 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.500524 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.500595 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.500676 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.500755 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.500829 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.500902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.500974 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.501046 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.501126 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.501206 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.501280 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.501399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.501476 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.501548 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.501630 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.501703 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.501776 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.501848 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.501921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.501992 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.502104 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.502182 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.502303 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.502387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.502461 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.502534 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.502615 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.502690 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.502762 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.502835 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.502906 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] auth.external = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.502985 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.503067 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.503141 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.503214 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.503287 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] auth.external = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.503470 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.503587 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.503709 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] auth.password = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.503839 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.503944 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] auth.token = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.504081 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.504187 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.504318 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] auth.password = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.504445 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.504555 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] auth.token = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.504667 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.504801 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.504905 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.505045 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.505156 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.505304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.505431 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.505547 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.505634 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.505803 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.505886 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] credential.caching = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.506093 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.506173 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.506283 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.506363 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.506528 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] credential.caching = True {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.506608 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.506783 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.506864 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.507034 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.507112 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.507876 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.507974 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.508065 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.508141 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.508301 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.508388 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.508547 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.508630 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.508806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.508893 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.509062 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.509141 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.509307 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.509389 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.509569 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.509651 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.509826 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.509907 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.510088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.510168 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.510319 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] federation.caching = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.510397 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.510556 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.510636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.510807 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.510886 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.511059 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.511140 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] federation.caching = True {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.511292 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.511447 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.511570 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.511671 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.511812 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.511896 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.512047 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.512128 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.512319 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.512398 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.512535 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.512617 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.512773 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.512853 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.513025 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.513107 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.513281 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] identity.caching = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.513403 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.513519 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.513635 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.513767 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.513926 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.514043 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.514146 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] identity.caching = True {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.514228 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.514349 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.514469 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.514551 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.514709 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.514819 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.514949 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.515031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.515201 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.515281 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.515473 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.515586 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.515958 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.516043 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.516201 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.516309 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.516430 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.516509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.516671 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.516769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.516918 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.517007 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.517156 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.517242 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.517411 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.517494 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.517658 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.517746 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.517912 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.517990 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.518161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.518280 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.518466 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.518551 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.518626 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.518699 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.518856 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.518944 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.519092 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.519181 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.519333 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.519443 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.519608 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.519689 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.519852 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.519944 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.520095 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.520174 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.520326 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.520412 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.520577 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.520655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.520827 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.520897 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.521063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.521147 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.521318 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.521464 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.521618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.521754 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.521888 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.522018 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.522175 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.522321 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.522445 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.522578 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.522721 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.523492 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.523577 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.523656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.523783 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.523869 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.523991 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.524114 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.524244 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.524381 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.524512 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.524596 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.524658 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.524717 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.524865 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.524967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.525101 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.525173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.525411 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.525540 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.525665 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.525798 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.525935 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.526105 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.526234 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.526315 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.526389 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.526473 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.526549 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.526623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.526726 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.526809 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.526989 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.527080 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.527225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.527308 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.user = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.527497 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.527583 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.527751 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.531661 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.531661 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.531937 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.531937 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.532078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.532153 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.532239 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.532322 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.532398 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.532472 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.532534 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.532657 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.532745 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.532822 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.532896 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.532970 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.533044 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.533126 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.533202 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.533276 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.533349 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.533422 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.533542 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.533630 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.533694 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.533799 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.533921 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.534000 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.534075 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.534158 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.534234 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.534307 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.534380 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.534454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.534527 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.534677 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.534757 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.534832 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.534906 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.534980 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.535053 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.535135 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.535211 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.535284 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.535359 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.535466 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.535545 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.535632 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.535709 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.535783 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.535856 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.535930 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.536003 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.536149 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.536227 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.536302 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.536376 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.536456 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.536532 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.536626 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.536762 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.536840 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.536914 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.536988 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] resource.caching = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.537062 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.537144 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.537220 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.537294 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.537367 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.537440 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.537513 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] resource.caching = True {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.537595 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.537671 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.537745 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.537818 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.537891 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.537964 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.538048 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.538124 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.538198 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.538198 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.538339 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.538433 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] role.caching = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.538520 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.538602 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] role.driver = sql {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.538678 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.538752 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.538825 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.538898 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.539011 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.539088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.539163 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] role.caching = True {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.539236 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.539309 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] role.driver = sql {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.539309 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.539490 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.539569 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.539643 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.539643 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.539776 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.539850 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.539931 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.540007 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.540080 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.540153 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.540226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.540355 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.540446 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.540604 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.540685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.540854 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.540936 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.541092 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.541172 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.541347 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.541527 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.541609 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.541775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.541907 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.541989 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.542063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.542244 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.542494 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.542575 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.542735 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.542824 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.542969 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.543050 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.543227 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.543308 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.543502 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.543588 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.543747 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.543827 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.543984 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.544065 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.544219 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.544303 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.544452 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.544540 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.544706 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.544788 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.544944 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.545026 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.545183 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.545265 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.545420 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.545501 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.545658 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.545739 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.545891 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.545973 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.546131 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] token.caching = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.546211 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.546370 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.546453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.546623 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.546704 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.546858 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.546938 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.547102 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] token.caching = True {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.547184 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.547345 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.547441 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.547605 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.547689 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.547878 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.547960 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.548170 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.548266 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.548343 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.548427 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.548576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.548659 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.548818 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.548899 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.549060 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.549156 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.549309 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.549390 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.549550 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.549629 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.549779 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.549868 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.550023 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.550103 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.550276 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.550373 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.550536 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.550679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.connection = **** {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.550773 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.550863 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.551009 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.551105 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.551264 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.551346 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.551532 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.551629 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] database.connection = **** {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.551773 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.551865 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.552014 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.552096 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.552255 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.552337 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.552494 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.552576 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.552743 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.552825 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.553011 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.553109 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.553260 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.553339 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.553453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.553535 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.553699 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.553780 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.553940 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.554022 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.554212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.554295 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.554447 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.554529 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.554692 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.554775 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.554931 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.555011 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.555106 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 18 12:00:20.555187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.555403 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.555681 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.555923 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 18 12:00:20.556099 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81863) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 18 12:00:20.584956 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.585187 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.585971 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.586380 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81863) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.586620 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81863) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.586854 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81863) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.594426 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.595380 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81863) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.635988 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.636220 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.636310 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81863) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.636517 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81863) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.705677 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.705802 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81863) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.758776 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.759022 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.759983 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81863) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.760224 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81863) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.925702 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.926029 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.926151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.926357 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.926579 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.926800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.930324 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81863) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.930562 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81863) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.930845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81863) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.931110 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81863) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.931292 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81863) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.931544 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81863) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.957582 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.957756 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81863) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.957891 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.957974 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81863) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.958052 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.958216 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81863) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.958302 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.958442 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81863) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.958537 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.958659 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.958743 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81863) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.958878 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81863) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.958961 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.959098 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81863) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.970282 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.970674 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81863) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.986938 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.987080 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81863) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.987184 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.987266 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81863) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.987389 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.987498 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81863) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.987643 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.987725 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81863) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.987868 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.987972 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81863) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.988090 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.988173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81863) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.988309 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81863) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:20.988404 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 18 12:00:21.005570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.005785 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 18 12:00:21.005870 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.006006 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 18 12:00:21.006139 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.006348 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 18 12:00:21.006430 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.006555 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 18 12:00:21.006636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.006840 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.006985 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.007116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.007451 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.007685 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.007757 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.007957 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.017724 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.018010 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.018291 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.018583 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.020279 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.025818 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.025818 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.025818 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.026026 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81862) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 12:00:21.028470 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81862) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 12:00:21.028584 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81863) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 12:00:21.028760 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.029049 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.029338 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.029631 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.029901 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.030178 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.030459 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.030848 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.031132 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.031409 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.031516 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81863) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 12:00:21.031741 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.032047 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.032326 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.032620 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.032895 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.033177 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.033479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.033876 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.034163 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.034447 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.055367 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.056861 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81862) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 12:00:21.057150 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.057980 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.059483 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81863) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 12:00:21.059776 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.061887 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81862) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 12:00:21.062179 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.062470 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.064450 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81863) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 12:00:21.064750 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.065045 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.068909 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81862) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 12:00:21.069200 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.071470 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81863) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 12:00:21.071768 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.073381 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81862) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 12:00:21.073681 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.073978 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.074348 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.074672 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.074933 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.075218 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.075527 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.075857 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81863) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 12:00:21.076157 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.077983 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.077983 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.077983 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.077983 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.077983 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.077983 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.092552 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.092841 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.094972 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.095262 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.096655 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81862) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 12:00:21.099216 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81863) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 12:00:21.099838 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.100150 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.103070 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.103375 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.104832 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81862) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 12:00:21.107925 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81862) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 12:00:21.110313 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81863) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 12:00:21.111280 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.111615 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.111907 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.112198 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.112482 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.112772 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.113585 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81863) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 12:00:21.117460 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.117991 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.118400 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.118699 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.118994 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.119286 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.129154 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81862) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 12:00:21.129462 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.129736 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.130016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.135285 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81863) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 12:00:21.135636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.135948 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.136187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.136898 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.138426 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.138723 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.141650 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81862) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 12:00:21.141937 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.142228 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.142533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.142814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.143365 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.144950 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.145323 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.147934 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81863) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 12:00:21.148269 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.148562 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.148871 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.149160 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.154125 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81862) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 12:00:21.154414 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.154701 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.154985 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.155272 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.155580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.155876 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.160531 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81863) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 12:00:21.160821 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.161112 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.161401 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.161692 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.161977 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.162274 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.171658 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81862) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 12:00:21.174685 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81862) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 12:00:21.177707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.178163 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81863) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 12:00:21.179692 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.181191 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81862) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 12:00:21.181483 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.181786 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81863) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 12:00:21.181929 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.184751 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.186206 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.187741 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81863) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 12:00:21.188035 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.188223 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.188355 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.189676 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81862) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 12:00:21.192677 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.192975 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.193265 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.193548 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.194886 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.196405 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81863) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 12:00:21.199339 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.199786 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.200110 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.200403 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.201510 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81862) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 12:00:21.201808 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.202210 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.208420 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81863) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 12:00:21.208669 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.208932 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81862) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 12:00:21.209066 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.209223 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.209509 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.209792 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.210070 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.210381 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.210668 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.210957 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.211230 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.211548 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.211838 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.212104 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.212382 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.212645 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.215919 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81863) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 18 12:00:21.216210 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.216501 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.216790 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.217073 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.217366 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.217648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.217924 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.218213 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.218507 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.218786 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.219056 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.219343 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.219644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81863) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 18 12:00:21.247932 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x559ab97d19c0 pid: 81862 (default app) Aug 18 12:00:21.251623 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0855ac-8e71-4656-a294-f24d63191233 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:21.251915 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0855ac-8e71-4656-a294-f24d63191233 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:21.252193 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0855ac-8e71-4656-a294-f24d63191233 None None] PATH_INFO: `/v3` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:21.256398 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1/1] 10.222.0.98 () {50 vars in 791 bytes} [Thu Aug 18 12:00:21 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:21.260845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x559ab97d19c0 pid: 81863 (default app) Aug 18 12:00:28.774605 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d729f3-828d-4880-891f-0a2736b982bb None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:28.775723 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d729f3-828d-4880-891f-0a2736b982bb None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:28.775816 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d729f3-828d-4880-891f-0a2736b982bb None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:28.777124 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 2/2] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:00:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 12:00:28.784767 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050fc64-73b0-4fa8-89a1-55dffd643129 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:28.785218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050fc64-73b0-4fa8-89a1-55dffd643129 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:28.785308 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050fc64-73b0-4fa8-89a1-55dffd643129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:28.854979 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: INFO dbcounter [None req-f050fc64-73b0-4fa8-89a1-55dffd643129 None None] Registered counter for database keystone Aug 18 12:00:28.876852 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG oslo_db.sqlalchemy.engines [None req-f050fc64-73b0-4fa8-89a1-55dffd643129 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81863) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 18 12:00:28.883185 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writer thread running {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 18 12:00:29.081626 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG passlib.handlers.bcrypt [None req-f050fc64-73b0-4fa8-89a1-55dffd643129 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81863) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 18 12:00:29.082028 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG passlib.handlers.bcrypt [None req-f050fc64-73b0-4fa8-89a1-55dffd643129 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81863) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 18 12:00:29.143469 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-f050fc64-73b0-4fa8-89a1-55dffd643129 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:00:29.145130 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-f050fc64-73b0-4fa8-89a1-55dffd643129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:00:29.275947 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1/3] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:00:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 494 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 18 12:00:29.762374 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d729f3-828d-4880-891f-0a2736b982bb None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:29.830677 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: INFO dbcounter [None req-04d729f3-828d-4880-891f-0a2736b982bb None None] Registered counter for database keystone Aug 18 12:00:29.850390 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG oslo_db.sqlalchemy.engines [None req-04d729f3-828d-4880-891f-0a2736b982bb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81862) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 18 12:00:29.857346 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writer thread running {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 18 12:00:29.981114 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d729f3-828d-4880-891f-0a2736b982bb None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:29.985475 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc84b0-0361-4221-bced-9dc4f64d5f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:29.986127 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc84b0-0361-4221-bced-9dc4f64d5f5c admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:29.986408 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc84b0-0361-4221-bced-9dc4f64d5f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:29.986685 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc84b0-0361-4221-bced-9dc4f64d5f5c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:29.994875 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dc84b0-0361-4221-bced-9dc4f64d5f5c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:30.046416 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f3dc84b0-0361-4221-bced-9dc4f64d5f5c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 18 12:00:30.046764 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dc84b0-0361-4221-bced-9dc4f64d5f5c admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:30.056959 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 3/4] 10.222.0.98 () {58 vars in 1324 bytes} [Thu Aug 18 12:00:29 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 12:00:30.062925 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f050fc64-73b0-4fa8-89a1-55dffd643129 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:30.077820 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f050fc64-73b0-4fa8-89a1-55dffd643129 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:30.082255 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c398fd-830c-4e0a-9ef1-e15b484641ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:30.083353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c398fd-830c-4e0a-9ef1-e15b484641ca admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:30.083667 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c398fd-830c-4e0a-9ef1-e15b484641ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:30.083943 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c398fd-830c-4e0a-9ef1-e15b484641ca admin admin] PATH_INFO: `/v3/projects/c192a2eba4e24841893715c8478b559c` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:30.085398 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c398fd-830c-4e0a-9ef1-e15b484641ca admin admin] RBAC: Authorizing `identity:get_project(project_id=c192a2eba4e24841893715c8478b559c)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:30.138157 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: WARNING keystone.common.rbac_enforcer.enforcer [None req-79c398fd-830c-4e0a-9ef1-e15b484641ca admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 18 12:00:30.138504 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c398fd-830c-4e0a-9ef1-e15b484641ca admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:30.143333 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 2/5] 10.222.0.98 () {56 vars in 1207 bytes} [Thu Aug 18 12:00:30 2022] GET /identity/v3/projects/c192a2eba4e24841893715c8478b559c => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:00:32.063717 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e65ff-542b-42f0-a470-d6728bdf2582 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:32.064331 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e65ff-542b-42f0-a470-d6728bdf2582 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:32.064331 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e65ff-542b-42f0-a470-d6728bdf2582 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:32.068407 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 4/6] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:00:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:00:32.073701 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf5a6e-aab6-49bc-b6b2-48a1e9486887 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:32.074139 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf5a6e-aab6-49bc-b6b2-48a1e9486887 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:32.074262 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf5a6e-aab6-49bc-b6b2-48a1e9486887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:32.094359 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-a5cf5a6e-aab6-49bc-b6b2-48a1e9486887 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:00:32.095930 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-a5cf5a6e-aab6-49bc-b6b2-48a1e9486887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:00:32.104636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 3/7] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:00:32 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 18 12:00:32.378885 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01e65ff-542b-42f0-a470-d6728bdf2582 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:32.391089 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01e65ff-542b-42f0-a470-d6728bdf2582 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:32.395579 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff531ce-25e3-41d9-b430-da76808415d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:32.396179 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff531ce-25e3-41d9-b430-da76808415d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:32.396465 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff531ce-25e3-41d9-b430-da76808415d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:32.396739 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff531ce-25e3-41d9-b430-da76808415d3 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:32.404348 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff531ce-25e3-41d9-b430-da76808415d3 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:32.406118 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff531ce-25e3-41d9-b430-da76808415d3 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:32.414753 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 5/8] 10.222.0.98 () {58 vars in 1324 bytes} [Thu Aug 18 12:00:32 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 12:00:32.420714 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf5a6e-aab6-49bc-b6b2-48a1e9486887 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:32.429765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf5a6e-aab6-49bc-b6b2-48a1e9486887 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:32.435880 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1370f56-89e8-4d17-a2c1-c8304df9eed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:32.436518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1370f56-89e8-4d17-a2c1-c8304df9eed0 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:32.436806 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1370f56-89e8-4d17-a2c1-c8304df9eed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:32.437086 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1370f56-89e8-4d17-a2c1-c8304df9eed0 admin admin] PATH_INFO: `/v3/users/6c92bf2b004241b3b54d40ae3aaaefe8` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:32.438410 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1370f56-89e8-4d17-a2c1-c8304df9eed0 admin admin] RBAC: Authorizing `identity:get_user(user_id=6c92bf2b004241b3b54d40ae3aaaefe8)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:32.440313 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1370f56-89e8-4d17-a2c1-c8304df9eed0 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:32.441847 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 4/9] 10.222.0.98 () {56 vars in 1198 bytes} [Thu Aug 18 12:00:32 2022] GET /identity/v3/users/6c92bf2b004241b3b54d40ae3aaaefe8 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:34.418362 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72acc99d-0ce2-40b4-bddd-12eecbf7e483 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:34.418974 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72acc99d-0ce2-40b4-bddd-12eecbf7e483 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:34.418974 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72acc99d-0ce2-40b4-bddd-12eecbf7e483 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:34.420289 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 6/10] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:00:34.425574 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3fcfd5-6777-462c-9495-e15510ad0fb8 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:34.425725 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20459c29-7314-465a-ad24-365c91b8c369 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:34.426126 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3fcfd5-6777-462c-9495-e15510ad0fb8 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:34.426219 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20459c29-7314-465a-ad24-365c91b8c369 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:34.426305 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3fcfd5-6777-462c-9495-e15510ad0fb8 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:34.426390 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20459c29-7314-465a-ad24-365c91b8c369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:34.427287 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 7/11] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:00:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 12:00:34.432977 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd503fef-e9dd-4460-bc36-f202b76b70f6 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:34.433263 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd503fef-e9dd-4460-bc36-f202b76b70f6 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:34.433536 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd503fef-e9dd-4460-bc36-f202b76b70f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:34.444565 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-20459c29-7314-465a-ad24-365c91b8c369 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:00:34.445755 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-20459c29-7314-465a-ad24-365c91b8c369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:00:34.474957 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG passlib.handlers.bcrypt [None req-bd503fef-e9dd-4460-bc36-f202b76b70f6 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81862) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 18 12:00:34.475397 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG passlib.handlers.bcrypt [None req-bd503fef-e9dd-4460-bc36-f202b76b70f6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81862) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 18 12:00:34.521668 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-bd503fef-e9dd-4460-bc36-f202b76b70f6 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:00:34.523278 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-bd503fef-e9dd-4460-bc36-f202b76b70f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:00:34.527216 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 5/12] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:00:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:00:34.621592 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 8/13] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:00:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 12:00:34.824788 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20459c29-7314-465a-ad24-365c91b8c369 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:34.848888 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:34.899089 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd503fef-e9dd-4460-bc36-f202b76b70f6 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:34.937058 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20459c29-7314-465a-ad24-365c91b8c369 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:34.970681 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1286c614-30d3-44f1-8888-b432569df9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:34.971325 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286c614-30d3-44f1-8888-b432569df9e8 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:34.971643 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286c614-30d3-44f1-8888-b432569df9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:34.971923 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286c614-30d3-44f1-8888-b432569df9e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:34.984202 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1286c614-30d3-44f1-8888-b432569df9e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:00:34.992676 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-1286c614-30d3-44f1-8888-b432569df9e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:00:34.992676 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:00:34.992676 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:00:34.992676 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:00:34.992676 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:00:34.992676 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:00:34.992676 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:00:34.992676 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:00:34.992676 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:00:34.992676 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:00:34.992676 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:00:34.992676 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:00:34.992676 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:00:34.992676 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:00:34.992676 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:00:34.992676 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:00:34.993800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:00:34.993800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:00:34.993800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:00:34.993800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:00:34.993800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:00:34.993800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:00:34.993800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:00:34.993800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:00:34.993800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:00:34.993800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:00:34.993800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:00:34.993800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:00:34.993800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:00:34.993800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:00:34.993800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:00:34.993800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:00:34.994644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:00:34.994644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:00:34.994644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:00:34.994644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:00:34.994644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:00:34.994644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:00:34.994644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:00:34.994644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:00:34.994644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:00:34.994644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:00:34.994644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:00:34.994644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:00:34.994644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 6/14] 10.222.0.98 () {56 vars in 1096 bytes} [Thu Aug 18 12:00:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:00:34.999638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1286c614-30d3-44f1-8888-b432569df9e8 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:35.008224 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd503fef-e9dd-4460-bc36-f202b76b70f6 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:35.011121 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:35.058796 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11887c-ad68-4df2-baba-cf6656ebb738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:35.059699 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11887c-ad68-4df2-baba-cf6656ebb738 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:35.059990 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11887c-ad68-4df2-baba-cf6656ebb738 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:35.060359 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11887c-ad68-4df2-baba-cf6656ebb738 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:35.084427 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:35.105975 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1286c614-30d3-44f1-8888-b432569df9e8 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:35.139079 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d11887c-ad68-4df2-baba-cf6656ebb738 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:35.139681 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290aebed-0227-4648-a738-52d855682c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:35.140304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290aebed-0227-4648-a738-52d855682c1e None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:35.140593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290aebed-0227-4648-a738-52d855682c1e None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:35.140688 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d11887c-ad68-4df2-baba-cf6656ebb738 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:35.140875 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290aebed-0227-4648-a738-52d855682c1e None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:35.141546 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290aebed-0227-4648-a738-52d855682c1e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:00:35.142111 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290aebed-0227-4648-a738-52d855682c1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:35.143523 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290aebed-0227-4648-a738-52d855682c1e None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:35.155791 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 7/15] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:00:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:00:35.161352 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290aebed-0227-4648-a738-52d855682c1e None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:35.242201 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 9/16] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:00:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 18 12:00:35.247823 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11887c-ad68-4df2-baba-cf6656ebb738 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:35.251033 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:35.263040 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290aebed-0227-4648-a738-52d855682c1e None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:35.297299 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4173bb-2656-4600-80e9-efcc3dbf5bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:35.297936 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4173bb-2656-4600-80e9-efcc3dbf5bdb None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:35.298220 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4173bb-2656-4600-80e9-efcc3dbf5bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:35.298502 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4173bb-2656-4600-80e9-efcc3dbf5bdb None admin] PATH_INFO: `/v3/users/6c92bf2b004241b3b54d40ae3aaaefe8` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:35.299726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4173bb-2656-4600-80e9-efcc3dbf5bdb None admin] RBAC: Authorizing `identity:get_user(user_id=6c92bf2b004241b3b54d40ae3aaaefe8)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:35.301205 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4173bb-2656-4600-80e9-efcc3dbf5bdb None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:35.302792 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 8/17] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:00:35 2022] GET /identity/v3/users/6c92bf2b004241b3b54d40ae3aaaefe8 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:00:35.308418 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4173bb-2656-4600-80e9-efcc3dbf5bdb None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:35.326861 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:35.350223 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11887c-ad68-4df2-baba-cf6656ebb738 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:35.383654 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55b450-ba31-48aa-a8ae-e5b8e6afc1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:35.384512 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55b450-ba31-48aa-a8ae-e5b8e6afc1be None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:35.385357 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55b450-ba31-48aa-a8ae-e5b8e6afc1be None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:35.385820 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55b450-ba31-48aa-a8ae-e5b8e6afc1be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:35.405869 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b55b450-ba31-48aa-a8ae-e5b8e6afc1be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:35.407346 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b55b450-ba31-48aa-a8ae-e5b8e6afc1be None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:35.411057 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4173bb-2656-4600-80e9-efcc3dbf5bdb None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:35.425520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-8b55b450-ba31-48aa-a8ae-e5b8e6afc1be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:00:35.425520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:00:35.425520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:00:35.425520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:00:35.425520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:00:35.425520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:00:35.425520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:00:35.425520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:00:35.425520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:00:35.425520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:00:35.425520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:00:35.425520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:00:35.425520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:00:35.425520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:00:35.425520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:00:35.425520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:00:35.426800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:00:35.426800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:00:35.426800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:00:35.426800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:00:35.426800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:00:35.426800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:00:35.426800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:00:35.426800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:00:35.426800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:00:35.426800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:00:35.426800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:00:35.426800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:00:35.426800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:00:35.426800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:00:35.426800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:00:35.426800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:00:35.426800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:00:35.427648 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:00:35.427648 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:00:35.427648 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:00:35.427648 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 10/18] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:00:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:00:35.432390 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55b450-ba31-48aa-a8ae-e5b8e6afc1be None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:35.445384 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2380f0-8f5b-4033-bd3c-d3b28c645dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:35.446032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2380f0-8f5b-4033-bd3c-d3b28c645dbe None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:35.446318 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2380f0-8f5b-4033-bd3c-d3b28c645dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:35.446597 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2380f0-8f5b-4033-bd3c-d3b28c645dbe None admin] PATH_INFO: `/v3/domains/default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:35.448331 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2380f0-8f5b-4033-bd3c-d3b28c645dbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:35.449812 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2380f0-8f5b-4033-bd3c-d3b28c645dbe None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:35.451383 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 9/19] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:00:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:00:35.463949 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2380f0-8f5b-4033-bd3c-d3b28c645dbe None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:35.493328 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:35.543179 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55b450-ba31-48aa-a8ae-e5b8e6afc1be None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:35.551097 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:35.572078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2380f0-8f5b-4033-bd3c-d3b28c645dbe None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:35.582329 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c12276-44eb-4c9b-9e26-e7a3b923313a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:35.582944 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c12276-44eb-4c9b-9e26-e7a3b923313a None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:35.583238 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c12276-44eb-4c9b-9e26-e7a3b923313a None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:35.583540 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c12276-44eb-4c9b-9e26-e7a3b923313a None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:35.584179 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c12276-44eb-4c9b-9e26-e7a3b923313a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:00:35.584480 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c12276-44eb-4c9b-9e26-e7a3b923313a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:35.585866 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c12276-44eb-4c9b-9e26-e7a3b923313a None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:35.604987 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 11/20] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:00:35.609388 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2bad9-3d0a-4172-9fd3-6fd5c6991a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:35.610025 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2bad9-3d0a-4172-9fd3-6fd5c6991a27 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:35.610312 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2bad9-3d0a-4172-9fd3-6fd5c6991a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:35.610603 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2bad9-3d0a-4172-9fd3-6fd5c6991a27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:35.611617 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2bad9-3d0a-4172-9fd3-6fd5c6991a27 None admin] RBAC: Adding query filter params (role.id=c53a0e13b58642f291c856efe1f0b87d, scope.domain.id=default, user.id=6c92bf2b004241b3b54d40ae3aaaefe8) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:00:35.611885 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2bad9-3d0a-4172-9fd3-6fd5c6991a27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:35.613384 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2bad9-3d0a-4172-9fd3-6fd5c6991a27 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:35.621747 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 10/21] 10.222.0.98 () {56 vars in 1322 bytes} [Thu Aug 18 12:00:35 2022] GET /identity/v3/role_assignments?user.id=6c92bf2b004241b3b54d40ae3aaaefe8&scope.domain.id=default&role.id=c53a0e13b58642f291c856efe1f0b87d => generated 233 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:00:37.805508 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52af769-dd30-4600-8bbe-bf4e7dc0242d None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:37.806030 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52af769-dd30-4600-8bbe-bf4e7dc0242d None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:37.806103 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52af769-dd30-4600-8bbe-bf4e7dc0242d None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:37.808105 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 12/22] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:00:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:00:37.813403 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4a0c0-4f93-4535-b492-1d89c6aaed78 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:37.814273 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4a0c0-4f93-4535-b492-1d89c6aaed78 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:37.814755 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4a0c0-4f93-4535-b492-1d89c6aaed78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:37.818881 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5c72e-200a-4a10-b841-bf43c2a57e9d None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:37.819788 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5c72e-200a-4a10-b841-bf43c2a57e9d None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:37.820804 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5c72e-200a-4a10-b841-bf43c2a57e9d None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:37.825740 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 13/23] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:00:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:00:37.839157 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71e60e-6890-41d6-95c6-c6db3340ae07 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:37.839736 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71e60e-6890-41d6-95c6-c6db3340ae07 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:37.840228 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71e60e-6890-41d6-95c6-c6db3340ae07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:37.849434 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-d1f4a0c0-4f93-4535-b492-1d89c6aaed78 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:00:37.852398 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-d1f4a0c0-4f93-4535-b492-1d89c6aaed78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:00:37.874676 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-fb71e60e-6890-41d6-95c6-c6db3340ae07 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:00:37.876209 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-fb71e60e-6890-41d6-95c6-c6db3340ae07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:00:38.001396 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 11/24] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:00:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:00:38.051102 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 14/25] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:00:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 12:00:38.195758 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a91fc9-bae6-45ab-b10f-0a4bdffaf33c None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:38.196073 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a91fc9-bae6-45ab-b10f-0a4bdffaf33c None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:38.196329 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a91fc9-bae6-45ab-b10f-0a4bdffaf33c None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:38.198034 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 12/26] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:00:38.203761 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00195d-b0b1-459e-97cb-718e554c1fab None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:38.204335 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00195d-b0b1-459e-97cb-718e554c1fab None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:38.204811 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00195d-b0b1-459e-97cb-718e554c1fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:38.232493 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-9c00195d-b0b1-459e-97cb-718e554c1fab None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:00:38.234049 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-9c00195d-b0b1-459e-97cb-718e554c1fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:00:38.289081 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:38.321213 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 15/27] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:00:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:00:38.488891 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a91fc9-bae6-45ab-b10f-0a4bdffaf33c None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:38.597459 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:38.633115 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a91fc9-bae6-45ab-b10f-0a4bdffaf33c None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:38.646719 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00195d-b0b1-459e-97cb-718e554c1fab None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:38.710920 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67eb3a-5030-4ad7-96f5-fe91757c6fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:38.711596 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67eb3a-5030-4ad7-96f5-fe91757c6fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:38.711881 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67eb3a-5030-4ad7-96f5-fe91757c6fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:38.712159 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67eb3a-5030-4ad7-96f5-fe91757c6fd4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:38.733093 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa67eb3a-5030-4ad7-96f5-fe91757c6fd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:00:38.737974 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-fa67eb3a-5030-4ad7-96f5-fe91757c6fd4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:00:38.737974 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:00:38.737974 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:00:38.737974 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:00:38.737974 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:00:38.737974 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:00:38.737974 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:00:38.737974 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:00:38.737974 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:00:38.737974 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:00:38.737974 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:00:38.737974 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:00:38.737974 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:00:38.737974 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:00:38.737974 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:00:38.737974 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:00:38.738800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:00:38.738800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:00:38.738800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:00:38.738800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:00:38.738800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:00:38.738800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:00:38.738800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:00:38.738800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:00:38.738800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:00:38.738800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:00:38.738800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:00:38.738800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:00:38.738800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:00:38.738800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:00:38.738800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:00:38.738800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:00:38.739578 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:00:38.739578 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:00:38.739578 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:00:38.739578 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:00:38.739578 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:00:38.739578 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:00:38.739578 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:00:38.739578 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:00:38.739578 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:00:38.739578 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:00:38.739578 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:00:38.739578 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:00:38.739578 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 13/28] 10.222.0.98 () {56 vars in 1096 bytes} [Thu Aug 18 12:00:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:00:38.740224 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67eb3a-5030-4ad7-96f5-fe91757c6fd4 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:38.822937 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00195d-b0b1-459e-97cb-718e554c1fab None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:38.882060 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41695f2d-584f-441d-8bd6-23a8488ec263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:38.882630 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41695f2d-584f-441d-8bd6-23a8488ec263 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:38.882914 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41695f2d-584f-441d-8bd6-23a8488ec263 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:38.883189 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41695f2d-584f-441d-8bd6-23a8488ec263 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:38.901852 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41695f2d-584f-441d-8bd6-23a8488ec263 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:38.903348 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41695f2d-584f-441d-8bd6-23a8488ec263 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:38.934089 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-41695f2d-584f-441d-8bd6-23a8488ec263 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:00:38.934089 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:00:38.934089 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:00:38.934089 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:00:38.934089 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:00:38.934089 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:00:38.934089 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:00:38.934089 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:00:38.934089 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:00:38.934089 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:00:38.934089 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:00:38.934089 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:00:38.934089 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:00:38.934089 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:00:38.934089 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:00:38.934089 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:00:38.935016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:00:38.935016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:00:38.935016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:00:38.935016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:00:38.935016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:00:38.935016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:00:38.935016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:00:38.935016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:00:38.935016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:00:38.935016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:00:38.935016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:00:38.935016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:00:38.935016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:00:38.935016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:00:38.935016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:00:38.935016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:00:38.935016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:00:38.935879 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:00:38.935879 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:00:38.935879 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:00:38.935879 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 16/29] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:00:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:00:38.936080 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67eb3a-5030-4ad7-96f5-fe91757c6fd4 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:38.936469 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41695f2d-584f-441d-8bd6-23a8488ec263 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:38.972771 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:38.976911 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab0bb9-a263-428e-91b6-0d75e3694e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:38.977546 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab0bb9-a263-428e-91b6-0d75e3694e11 None admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:38.977872 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab0bb9-a263-428e-91b6-0d75e3694e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:38.978161 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab0bb9-a263-428e-91b6-0d75e3694e11 None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:38.978876 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bab0bb9-a263-428e-91b6-0d75e3694e11 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:38.980385 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bab0bb9-a263-428e-91b6-0d75e3694e11 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:38.992513 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:39.053031 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41695f2d-584f-441d-8bd6-23a8488ec263 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:39.090472 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc3f651-89eb-4ce8-8371-ec9944d42057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:39.091092 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3f651-89eb-4ce8-8371-ec9944d42057 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:39.091376 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3f651-89eb-4ce8-8371-ec9944d42057 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:39.091681 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3f651-89eb-4ce8-8371-ec9944d42057 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:39.092359 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc3f651-89eb-4ce8-8371-ec9944d42057 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:00:39.092707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc3f651-89eb-4ce8-8371-ec9944d42057 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:39.094098 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc3f651-89eb-4ce8-8371-ec9944d42057 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:39.109037 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 17/30] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:00:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:39.110493 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13adfe02-408e-4bfd-b990-e3a256016ab9 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:39.110793 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13adfe02-408e-4bfd-b990-e3a256016ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:39.114484 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13adfe02-408e-4bfd-b990-e3a256016ab9 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:39.114484 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 18/31] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:00:39.114484 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13adfe02-408e-4bfd-b990-e3a256016ab9 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:39.220991 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:39.232839 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13adfe02-408e-4bfd-b990-e3a256016ab9 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:39.233164 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 14/32] 10.222.0.98 () {60 vars in 1131 bytes} [Thu Aug 18 12:00:38 2022] POST /identity/v3/roles => generated 219 bytes in 493 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:00:39.233952 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab0bb9-a263-428e-91b6-0d75e3694e11 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:39.303664 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b97c7-8aa6-4f56-8287-d2780922ee6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:39.304351 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b97c7-8aa6-4f56-8287-d2780922ee6f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:39.304641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b97c7-8aa6-4f56-8287-d2780922ee6f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:39.304920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b97c7-8aa6-4f56-8287-d2780922ee6f None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:39.305640 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4b97c7-8aa6-4f56-8287-d2780922ee6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:00:39.305959 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4b97c7-8aa6-4f56-8287-d2780922ee6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:39.307971 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4b97c7-8aa6-4f56-8287-d2780922ee6f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:39.322160 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 19/33] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:00:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:39.324238 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d910c-389f-42ae-9b3e-cab558c942c1 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:39.324530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d910c-389f-42ae-9b3e-cab558c942c1 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:39.324806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d910c-389f-42ae-9b3e-cab558c942c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:39.345083 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-608d910c-389f-42ae-9b3e-cab558c942c1 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:00:39.346377 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-608d910c-389f-42ae-9b3e-cab558c942c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:00:39.383536 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab0bb9-a263-428e-91b6-0d75e3694e11 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:39.424921 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28236c9c-7fbb-4508-9fd9-37c48e3738ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:39.425735 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28236c9c-7fbb-4508-9fd9-37c48e3738ca None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:39.426118 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28236c9c-7fbb-4508-9fd9-37c48e3738ca None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:39.426499 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28236c9c-7fbb-4508-9fd9-37c48e3738ca None admin] PATH_INFO: `/v3/users/6c92bf2b004241b3b54d40ae3aaaefe8` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:39.427896 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28236c9c-7fbb-4508-9fd9-37c48e3738ca None admin] RBAC: Authorizing `identity:get_user(user_id=6c92bf2b004241b3b54d40ae3aaaefe8)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:39.428300 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 20/34] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:00:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:00:39.429579 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f6f01-28df-4dc3-855b-c83b9d694ae1 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:39.429768 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28236c9c-7fbb-4508-9fd9-37c48e3738ca None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:39.429904 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f6f01-28df-4dc3-855b-c83b9d694ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:39.430144 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f6f01-28df-4dc3-855b-c83b9d694ae1 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:39.431372 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 21/35] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:00:39.433197 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f6f01-28df-4dc3-855b-c83b9d694ae1 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:39.436649 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 15/36] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:00:39 2022] GET /identity/v3/users/6c92bf2b004241b3b54d40ae3aaaefe8 => generated 247 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:39.438131 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a5999-625d-4c2d-97fa-3cf31446c0a5 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:39.438427 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a5999-625d-4c2d-97fa-3cf31446c0a5 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:39.438712 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a5999-625d-4c2d-97fa-3cf31446c0a5 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:39.440167 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 16/37] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:00:39.440914 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1a5999-625d-4c2d-97fa-3cf31446c0a5 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:39.514229 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:39.541414 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:39.557819 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1a5999-625d-4c2d-97fa-3cf31446c0a5 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:39.564199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f6f01-28df-4dc3-855b-c83b9d694ae1 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:39.594768 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8546d-af13-4be2-a7aa-c9453dbdf095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:39.595496 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8546d-af13-4be2-a7aa-c9453dbdf095 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:39.595789 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8546d-af13-4be2-a7aa-c9453dbdf095 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:39.596125 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8546d-af13-4be2-a7aa-c9453dbdf095 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:39.597362 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be8546d-af13-4be2-a7aa-c9453dbdf095 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:39.598819 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be8546d-af13-4be2-a7aa-c9453dbdf095 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:39.600717 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 17/38] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:00:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:39.605940 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734cde1-514e-485a-a007-41eaae4d21f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:39.606333 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a7059-16e8-4eb4-8c5f-5893c70b74c3 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:39.606629 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a7059-16e8-4eb4-8c5f-5893c70b74c3 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:39.606916 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a7059-16e8-4eb4-8c5f-5893c70b74c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:39.607360 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734cde1-514e-485a-a007-41eaae4d21f9 None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:39.607849 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734cde1-514e-485a-a007-41eaae4d21f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:39.608299 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734cde1-514e-485a-a007-41eaae4d21f9 None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:39.609360 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e734cde1-514e-485a-a007-41eaae4d21f9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:39.610958 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e734cde1-514e-485a-a007-41eaae4d21f9 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:39.626028 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-2d9a7059-16e8-4eb4-8c5f-5893c70b74c3 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:00:39.627620 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-2d9a7059-16e8-4eb4-8c5f-5893c70b74c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:00:39.700870 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 18/39] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:00:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:00:39.702243 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd48b8-96db-4361-90a3-25a06277467a None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:39.702535 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd48b8-96db-4361-90a3-25a06277467a None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:39.702814 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd48b8-96db-4361-90a3-25a06277467a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:39.720288 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-9fcd48b8-96db-4361-90a3-25a06277467a None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:00:39.721473 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-9fcd48b8-96db-4361-90a3-25a06277467a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:00:39.794975 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:39.808595 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 19/40] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:00:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:00:39.812507 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd48b8-96db-4361-90a3-25a06277467a None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:39.822899 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 22/41] 10.222.0.98 () {60 vars in 1141 bytes} [Thu Aug 18 12:00:39 2022] POST /identity/v3/projects => generated 301 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:00:39.823904 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734cde1-514e-485a-a007-41eaae4d21f9 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:39.939793 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734cde1-514e-485a-a007-41eaae4d21f9 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:39.964077 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd48b8-96db-4361-90a3-25a06277467a None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:39.978546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:39.986278 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30db332-f9dd-496b-8768-5642c012b5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:39.989230 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30db332-f9dd-496b-8768-5642c012b5fe None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:39.989230 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30db332-f9dd-496b-8768-5642c012b5fe None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:39.989230 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30db332-f9dd-496b-8768-5642c012b5fe None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:39.989230 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30db332-f9dd-496b-8768-5642c012b5fe None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:39.990212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30db332-f9dd-496b-8768-5642c012b5fe None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:40.023459 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a312b7-d624-4681-9e3e-015f360dbabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:40.023459 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a312b7-d624-4681-9e3e-015f360dbabc None admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:40.023459 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a312b7-d624-4681-9e3e-015f360dbabc None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:40.023459 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a312b7-d624-4681-9e3e-015f360dbabc None admin] PATH_INFO: `/v3/domains/default/users/6c92bf2b004241b3b54d40ae3aaaefe8/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:40.023871 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a312b7-d624-4681-9e3e-015f360dbabc None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=6c92bf2b004241b3b54d40ae3aaaefe8, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:40.023871 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a312b7-d624-4681-9e3e-015f360dbabc None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:40.080034 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 23/42] 10.222.0.98 () {60 vars in 1131 bytes} [Thu Aug 18 12:00:39 2022] POST /identity/v3/roles => generated 225 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:00:40.081206 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30db332-f9dd-496b-8768-5642c012b5fe None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:40.124212 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 20/43] 10.222.0.98 () {58 vars in 1361 bytes} [Thu Aug 18 12:00:39 2022] PUT /identity/v3/domains/default/users/6c92bf2b004241b3b54d40ae3aaaefe8/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:00:40.208808 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30db332-f9dd-496b-8768-5642c012b5fe None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:40.209733 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a312b7-d624-4681-9e3e-015f360dbabc None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:40.288322 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84bd44-c06d-411e-88de-923b406b7d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:40.289407 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84bd44-c06d-411e-88de-923b406b7d65 None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:40.289952 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84bd44-c06d-411e-88de-923b406b7d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:40.290240 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84bd44-c06d-411e-88de-923b406b7d65 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:40.290960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb84bd44-c06d-411e-88de-923b406b7d65 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:40.292541 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb84bd44-c06d-411e-88de-923b406b7d65 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:40.317254 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:40.380286 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 24/44] 10.222.0.98 () {60 vars in 1131 bytes} [Thu Aug 18 12:00:40 2022] POST /identity/v3/roles => generated 223 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:00:40.399377 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a312b7-d624-4681-9e3e-015f360dbabc None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:40.441884 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be8ede5-0ddb-4505-baf5-4518b3095b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:40.443002 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8ede5-0ddb-4505-baf5-4518b3095b84 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:40.443670 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8ede5-0ddb-4505-baf5-4518b3095b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:40.444157 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8ede5-0ddb-4505-baf5-4518b3095b84 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:40.445826 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be8ede5-0ddb-4505-baf5-4518b3095b84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:40.447860 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be8ede5-0ddb-4505-baf5-4518b3095b84 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:40.450616 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 21/45] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:00:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:00:40.457365 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84bd44-c06d-411e-88de-923b406b7d65 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:40.524434 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:40.577315 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84bd44-c06d-411e-88de-923b406b7d65 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:40.613073 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25d4ac-a95d-4ec7-bd52-f241bbe4e75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:40.613706 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25d4ac-a95d-4ec7-bd52-f241bbe4e75d None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:40.613992 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25d4ac-a95d-4ec7-bd52-f241bbe4e75d None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:40.614273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25d4ac-a95d-4ec7-bd52-f241bbe4e75d None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:40.614946 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25d4ac-a95d-4ec7-bd52-f241bbe4e75d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:40.616484 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25d4ac-a95d-4ec7-bd52-f241bbe4e75d None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:40.741026 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 25/46] 10.222.0.98 () {60 vars in 1141 bytes} [Thu Aug 18 12:00:40 2022] POST /identity/v3/projects => generated 312 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:00:42.141072 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3af6eb6-cf20-40aa-b288-14f11cc6ac8a None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:42.141611 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3af6eb6-cf20-40aa-b288-14f11cc6ac8a None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:42.141700 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3af6eb6-cf20-40aa-b288-14f11cc6ac8a None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:42.143285 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 22/47] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:00:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:00:42.148604 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe469061-d729-4bde-be1c-7e286324964a None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:42.149002 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe469061-d729-4bde-be1c-7e286324964a None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:42.149240 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe469061-d729-4bde-be1c-7e286324964a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:42.167746 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-fe469061-d729-4bde-be1c-7e286324964a None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:00:42.169257 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-fe469061-d729-4bde-be1c-7e286324964a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:00:42.235106 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 26/48] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:00:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:00:42.499446 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3af6eb6-cf20-40aa-b288-14f11cc6ac8a None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:42.661320 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9666a0f-bb8c-484d-846f-86488a6fb8cd None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:42.661730 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9666a0f-bb8c-484d-846f-86488a6fb8cd None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:42.662099 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9666a0f-bb8c-484d-846f-86488a6fb8cd None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:42.663504 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 27/49] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:00:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 12:00:42.669647 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19c2a7-9266-491c-8891-2cf972e744a4 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:42.669931 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19c2a7-9266-491c-8891-2cf972e744a4 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:42.670217 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19c2a7-9266-491c-8891-2cf972e744a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:42.688290 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-4c19c2a7-9266-491c-8891-2cf972e744a4 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:00:42.689537 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-4c19c2a7-9266-491c-8891-2cf972e744a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:00:42.709697 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3af6eb6-cf20-40aa-b288-14f11cc6ac8a None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:42.718366 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:42.729141 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:42.745741 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257f823-9e55-4346-9556-748bfc791f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:42.746378 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257f823-9e55-4346-9556-748bfc791f58 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:42.746665 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257f823-9e55-4346-9556-748bfc791f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:42.746949 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257f823-9e55-4346-9556-748bfc791f58 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:42.757142 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5257f823-9e55-4346-9556-748bfc791f58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:00:42.757919 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 28/50] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:00:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 12:00:42.762838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-5257f823-9e55-4346-9556-748bfc791f58 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:00:42.762838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:00:42.762838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:00:42.762838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:00:42.762838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:00:42.762838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:00:42.762838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:00:42.762838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:00:42.762838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:00:42.762838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:00:42.762838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:00:42.762838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:00:42.762838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:00:42.762838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:00:42.762838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:00:42.762838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:00:42.768967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:00:42.768967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:00:42.768967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:00:42.768967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:00:42.768967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:00:42.768967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:00:42.768967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:00:42.768967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:00:42.768967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:00:42.768967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:00:42.768967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:00:42.768967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:00:42.768967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:00:42.768967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:00:42.768967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:00:42.768967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:00:42.769734 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:00:42.769734 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:00:42.769734 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:00:42.769734 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:00:42.769734 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:00:42.769734 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:00:42.769734 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:00:42.769734 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:00:42.769734 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:00:42.769734 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:00:42.769734 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:00:42.769734 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:00:42.769734 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 23/51] 10.222.0.98 () {56 vars in 1096 bytes} [Thu Aug 18 12:00:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:00:42.770346 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19c2a7-9266-491c-8891-2cf972e744a4 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:42.867865 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19c2a7-9266-491c-8891-2cf972e744a4 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:42.902918 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e754f-38a7-4410-8e8a-b835fe57cde2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:42.903684 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e754f-38a7-4410-8e8a-b835fe57cde2 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:42.903985 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e754f-38a7-4410-8e8a-b835fe57cde2 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:42.904267 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e754f-38a7-4410-8e8a-b835fe57cde2 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:42.904944 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1e754f-38a7-4410-8e8a-b835fe57cde2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:00:42.905257 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1e754f-38a7-4410-8e8a-b835fe57cde2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:42.906636 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1e754f-38a7-4410-8e8a-b835fe57cde2 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:42.914849 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 29/52] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:00:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:00:42.920643 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257f823-9e55-4346-9556-748bfc791f58 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:43.019348 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257f823-9e55-4346-9556-748bfc791f58 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:43.024112 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e754f-38a7-4410-8e8a-b835fe57cde2 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:43.054707 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c994b33-8981-4e54-9828-ff2c6c556dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:43.055372 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c994b33-8981-4e54-9828-ff2c6c556dfc None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:43.055691 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c994b33-8981-4e54-9828-ff2c6c556dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:43.055974 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c994b33-8981-4e54-9828-ff2c6c556dfc None admin] PATH_INFO: `/v3/users/6c92bf2b004241b3b54d40ae3aaaefe8` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:43.057723 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c994b33-8981-4e54-9828-ff2c6c556dfc None admin] RBAC: Authorizing `identity:get_user(user_id=6c92bf2b004241b3b54d40ae3aaaefe8)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:43.059254 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c994b33-8981-4e54-9828-ff2c6c556dfc None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:43.060987 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 24/53] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:00:42 2022] GET /identity/v3/users/6c92bf2b004241b3b54d40ae3aaaefe8 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:43.066878 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c994b33-8981-4e54-9828-ff2c6c556dfc None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:43.118213 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:43.160427 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:43.173095 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c994b33-8981-4e54-9828-ff2c6c556dfc None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:43.209400 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c62f40-4c26-4866-a0b2-4c11b305c041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:43.210053 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c62f40-4c26-4866-a0b2-4c11b305c041 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:43.210506 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c62f40-4c26-4866-a0b2-4c11b305c041 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:43.210629 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c62f40-4c26-4866-a0b2-4c11b305c041 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:43.211800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c62f40-4c26-4866-a0b2-4c11b305c041 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:43.213249 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c62f40-4c26-4866-a0b2-4c11b305c041 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:43.215382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 25/54] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:00:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:00:43.221196 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c62f40-4c26-4866-a0b2-4c11b305c041 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:43.237196 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e754f-38a7-4410-8e8a-b835fe57cde2 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:43.273266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae6b4cc-0f89-48b9-99d0-37ea9fd62a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:43.273837 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6b4cc-0f89-48b9-99d0-37ea9fd62a3f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:43.274128 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6b4cc-0f89-48b9-99d0-37ea9fd62a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:43.274408 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6b4cc-0f89-48b9-99d0-37ea9fd62a3f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:43.275597 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae6b4cc-0f89-48b9-99d0-37ea9fd62a3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:43.277587 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae6b4cc-0f89-48b9-99d0-37ea9fd62a3f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:43.279233 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 30/55] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:00:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:00:43.284804 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae6b4cc-0f89-48b9-99d0-37ea9fd62a3f None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:43.327285 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c62f40-4c26-4866-a0b2-4c11b305c041 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:43.341998 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:43.362585 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aea72f-6e00-4701-b3ad-24be38ea7ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:43.363294 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aea72f-6e00-4701-b3ad-24be38ea7ef5 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:43.363609 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aea72f-6e00-4701-b3ad-24be38ea7ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:43.363887 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aea72f-6e00-4701-b3ad-24be38ea7ef5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:43.364609 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aea72f-6e00-4701-b3ad-24be38ea7ef5 None admin] RBAC: Adding query filter params (role.id=c53a0e13b58642f291c856efe1f0b87d, scope.domain.id=default, user.id=6c92bf2b004241b3b54d40ae3aaaefe8) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:00:43.364918 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aea72f-6e00-4701-b3ad-24be38ea7ef5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:43.366361 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aea72f-6e00-4701-b3ad-24be38ea7ef5 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:43.372742 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 26/56] 10.222.0.98 () {56 vars in 1322 bytes} [Thu Aug 18 12:00:43 2022] GET /identity/v3/role_assignments?user.id=6c92bf2b004241b3b54d40ae3aaaefe8&scope.domain.id=default&role.id=c53a0e13b58642f291c856efe1f0b87d => generated 530 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:00:43.378763 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:43.386514 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae6b4cc-0f89-48b9-99d0-37ea9fd62a3f None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:43.424269 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296511f4-c52c-4741-85ad-4bb91c0d2990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:43.424920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296511f4-c52c-4741-85ad-4bb91c0d2990 None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:43.425213 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296511f4-c52c-4741-85ad-4bb91c0d2990 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:43.425494 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296511f4-c52c-4741-85ad-4bb91c0d2990 None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:43.426172 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296511f4-c52c-4741-85ad-4bb91c0d2990 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:43.427722 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296511f4-c52c-4741-85ad-4bb91c0d2990 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:43.519449 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 31/57] 10.222.0.98 () {60 vars in 1140 bytes} [Thu Aug 18 12:00:43 2022] POST /identity/v3/projects => generated 298 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:00:45.397319 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39d0f8-8764-477a-a1e2-3612a0d2dfd4 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:45.397868 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39d0f8-8764-477a-a1e2-3612a0d2dfd4 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:45.397868 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39d0f8-8764-477a-a1e2-3612a0d2dfd4 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:45.399203 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 27/58] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:00:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:00:45.404787 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9665dd4-c6ec-44bf-8831-d083cabc1461 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:45.405188 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9665dd4-c6ec-44bf-8831-d083cabc1461 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:45.405350 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9665dd4-c6ec-44bf-8831-d083cabc1461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:45.423598 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-a9665dd4-c6ec-44bf-8831-d083cabc1461 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:00:45.424912 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-a9665dd4-c6ec-44bf-8831-d083cabc1461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:00:45.489332 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 32/59] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:00:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:00:45.759538 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e39d0f8-8764-477a-a1e2-3612a0d2dfd4 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:45.861624 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e39d0f8-8764-477a-a1e2-3612a0d2dfd4 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:45.894785 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6edb28b-24e2-4c3a-a846-515fabc92f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:45.895437 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6edb28b-24e2-4c3a-a846-515fabc92f4b None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:45.895730 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6edb28b-24e2-4c3a-a846-515fabc92f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:45.896010 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6edb28b-24e2-4c3a-a846-515fabc92f4b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:45.897135 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6edb28b-24e2-4c3a-a846-515fabc92f4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:45.898585 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6edb28b-24e2-4c3a-a846-515fabc92f4b None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:45.900179 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 28/60] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:00:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:45.906308 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9665dd4-c6ec-44bf-8831-d083cabc1461 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:45.982235 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:46.004304 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9665dd4-c6ec-44bf-8831-d083cabc1461 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:46.038062 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ddf82e-8cd7-476d-92b6-b78c689ec140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:46.038669 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ddf82e-8cd7-476d-92b6-b78c689ec140 None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:46.038956 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ddf82e-8cd7-476d-92b6-b78c689ec140 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:46.039234 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ddf82e-8cd7-476d-92b6-b78c689ec140 None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:46.039933 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ddf82e-8cd7-476d-92b6-b78c689ec140 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:46.041659 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ddf82e-8cd7-476d-92b6-b78c689ec140 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:46.138222 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 33/61] 10.222.0.98 () {60 vars in 1132 bytes} [Thu Aug 18 12:00:45 2022] POST /identity/v3/users => generated 275 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:00:49.157318 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af628aad-326a-4ebe-a719-19f6d458666f None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:49.158209 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af628aad-326a-4ebe-a719-19f6d458666f None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:49.158636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af628aad-326a-4ebe-a719-19f6d458666f None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:49.160151 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 29/62] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:00:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:00:49.172969 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231da89e-3079-48ed-8243-bb4fdf94e4a0 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:49.173832 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231da89e-3079-48ed-8243-bb4fdf94e4a0 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:49.174284 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231da89e-3079-48ed-8243-bb4fdf94e4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:49.194157 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-231da89e-3079-48ed-8243-bb4fdf94e4a0 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:00:49.195614 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-231da89e-3079-48ed-8243-bb4fdf94e4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:00:49.296589 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 34/63] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:00:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:00:49.469183 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c24b6-384f-40b4-9883-67ddf12792b8 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:49.469743 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c24b6-384f-40b4-9883-67ddf12792b8 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:49.470170 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c24b6-384f-40b4-9883-67ddf12792b8 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:49.471754 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 30/64] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:00:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:00:49.479351 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb173c84-d38c-46cc-ab39-2a5194cccce8 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:49.480392 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb173c84-d38c-46cc-ab39-2a5194cccce8 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:49.480816 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb173c84-d38c-46cc-ab39-2a5194cccce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:49.504130 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-fb173c84-d38c-46cc-ab39-2a5194cccce8 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:00:49.505569 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-fb173c84-d38c-46cc-ab39-2a5194cccce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:00:49.538987 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f222e-a908-4fef-bb84-24ebba2ec8d4 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:49.539521 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f222e-a908-4fef-bb84-24ebba2ec8d4 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:49.539941 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f222e-a908-4fef-bb84-24ebba2ec8d4 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:49.541433 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 31/65] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:00:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 12:00:49.546852 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec15c50-60f4-4c26-a02b-e58c5606d317 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:49.547360 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec15c50-60f4-4c26-a02b-e58c5606d317 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:49.547843 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec15c50-60f4-4c26-a02b-e58c5606d317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:49.582832 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-aec15c50-60f4-4c26-a02b-e58c5606d317 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:00:49.584638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-aec15c50-60f4-4c26-a02b-e58c5606d317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:00:49.599813 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:49.618536 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 35/66] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:00:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:00:49.657736 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:49.669937 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb173c84-d38c-46cc-ab39-2a5194cccce8 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:49.701114 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 32/67] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:00:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 12:00:49.802222 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22459b-4715-48a2-8a01-544dc703542a None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:49.802756 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22459b-4715-48a2-8a01-544dc703542a None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:49.803184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22459b-4715-48a2-8a01-544dc703542a None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:49.804830 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 33/68] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:00:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 12:00:49.806734 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb173c84-d38c-46cc-ab39-2a5194cccce8 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:49.811476 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3294b6-2a6f-4a5f-b05c-ba4b65cc7a70 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:49.811986 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3294b6-2a6f-4a5f-b05c-ba4b65cc7a70 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:49.812516 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3294b6-2a6f-4a5f-b05c-ba4b65cc7a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:49.840178 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-8b3294b6-2a6f-4a5f-b05c-ba4b65cc7a70 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:00:49.843021 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-8b3294b6-2a6f-4a5f-b05c-ba4b65cc7a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:00:49.876645 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4371bd2-9478-4c03-b2b8-1d6d9f67b95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:49.877302 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4371bd2-9478-4c03-b2b8-1d6d9f67b95c None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:49.877586 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4371bd2-9478-4c03-b2b8-1d6d9f67b95c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:49.877870 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4371bd2-9478-4c03-b2b8-1d6d9f67b95c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:49.899098 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4371bd2-9478-4c03-b2b8-1d6d9f67b95c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:00:49.906161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-a4371bd2-9478-4c03-b2b8-1d6d9f67b95c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 12:00:49.906161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:00:49.906161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:00:49.906161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:00:49.906161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:00:49.906161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:00:49.906161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:00:49.906161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:00:49.906161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:00:49.906161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:00:49.906161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:00:49.906161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:00:49.906161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:00:49.906161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:00:49.906161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:00:49.906161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:00:49.907225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:00:49.907225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:00:49.907225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:00:49.907225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:00:49.907225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:00:49.907225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:00:49.907225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:00:49.907225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:00:49.907225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:00:49.907225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:00:49.907225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:00:49.907225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:00:49.907225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:00:49.907225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:00:49.907225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:00:49.907225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:00:49.908056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:00:49.908056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:00:49.908056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:00:49.908056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:00:49.908056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:00:49.908056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:00:49.908056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:00:49.908056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:00:49.908056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:00:49.908056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:00:49.908056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 12:00:49.908056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:00:49.908056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 36/69] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:00:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:00:49.908702 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759dbae1-2610-45c9-bb5e-1744ddfe1543 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:49.909009 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759dbae1-2610-45c9-bb5e-1744ddfe1543 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:49.910149 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759dbae1-2610-45c9-bb5e-1744ddfe1543 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:49.915448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 37/70] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:00:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:00:49.915833 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc3b8b-e4b9-499f-8b9d-2e07e5e540d0 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:49.916132 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc3b8b-e4b9-499f-8b9d-2e07e5e540d0 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:49.916418 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc3b8b-e4b9-499f-8b9d-2e07e5e540d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:49.934585 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-34dc3b8b-e4b9-499f-8b9d-2e07e5e540d0 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:00:49.937154 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-34dc3b8b-e4b9-499f-8b9d-2e07e5e540d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:00:49.954189 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 34/71] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:00:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:00:49.955829 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3294b6-2a6f-4a5f-b05c-ba4b65cc7a70 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:50.004070 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:50.036813 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 38/72] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:00:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:00:50.039228 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc3b8b-e4b9-499f-8b9d-2e07e5e540d0 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:50.069289 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3294b6-2a6f-4a5f-b05c-ba4b65cc7a70 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:50.077326 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:50.110975 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca2da0-55a6-4b53-81c8-59ca6efdaceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:50.112369 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca2da0-55a6-4b53-81c8-59ca6efdaceb None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:50.113081 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca2da0-55a6-4b53-81c8-59ca6efdaceb None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:50.113654 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca2da0-55a6-4b53-81c8-59ca6efdaceb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:50.127957 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ca2da0-55a6-4b53-81c8-59ca6efdaceb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:00:50.135184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-d3ca2da0-55a6-4b53-81c8-59ca6efdaceb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 12:00:50.135184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:00:50.135184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:00:50.135184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:00:50.135184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:00:50.135184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:00:50.135184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:00:50.135184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:00:50.135184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:00:50.135184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:00:50.135184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:00:50.135184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:00:50.135184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:00:50.135184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:00:50.135184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:00:50.135184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:00:50.136623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:00:50.136623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:00:50.136623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:00:50.136623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:00:50.136623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:00:50.136623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:00:50.136623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:00:50.136623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:00:50.136623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:00:50.136623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:00:50.136623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:00:50.136623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:00:50.136623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:00:50.136623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:00:50.136623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:00:50.136623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:00:50.137378 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:00:50.137378 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:00:50.137378 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:00:50.137378 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:00:50.137378 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:00:50.137378 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:00:50.137378 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:00:50.137378 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:00:50.137378 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:00:50.137378 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:00:50.137378 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 12:00:50.137378 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:00:50.139874 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 35/73] 10.222.0.98 () {56 vars in 1114 bytes} [Thu Aug 18 12:00:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:00:50.141867 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca2da0-55a6-4b53-81c8-59ca6efdaceb None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:50.158442 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc3b8b-e4b9-499f-8b9d-2e07e5e540d0 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:50.197588 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de76a7-ae01-41cf-a28a-920efb9f55fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:50.198386 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de76a7-ae01-41cf-a28a-920efb9f55fc None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:50.198892 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de76a7-ae01-41cf-a28a-920efb9f55fc None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:50.199485 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de76a7-ae01-41cf-a28a-920efb9f55fc None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:50.200362 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40de76a7-ae01-41cf-a28a-920efb9f55fc None admin] RBAC: Adding query filter params (name=member) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:00:50.200899 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40de76a7-ae01-41cf-a28a-920efb9f55fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:50.202511 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40de76a7-ae01-41cf-a28a-920efb9f55fc None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:50.211365 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 39/74] 10.222.0.98 () {56 vars in 1101 bytes} [Thu Aug 18 12:00:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:50.212516 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de76a7-ae01-41cf-a28a-920efb9f55fc None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:50.268351 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:50.280267 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca2da0-55a6-4b53-81c8-59ca6efdaceb None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:50.319477 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06493839-82d3-49d7-9d9a-0a58e8b6f7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:50.320177 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06493839-82d3-49d7-9d9a-0a58e8b6f7f3 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:50.320438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06493839-82d3-49d7-9d9a-0a58e8b6f7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:50.320677 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06493839-82d3-49d7-9d9a-0a58e8b6f7f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:50.330738 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de76a7-ae01-41cf-a28a-920efb9f55fc None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:50.331733 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06493839-82d3-49d7-9d9a-0a58e8b6f7f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:00:50.336969 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-06493839-82d3-49d7-9d9a-0a58e8b6f7f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:00:50.336969 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:00:50.336969 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:00:50.336969 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:00:50.336969 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:00:50.336969 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:00:50.336969 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:00:50.336969 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:00:50.336969 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:00:50.336969 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:00:50.336969 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:00:50.336969 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:00:50.336969 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:00:50.336969 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:00:50.336969 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:00:50.336969 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:00:50.337962 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:00:50.337962 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:00:50.337962 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:00:50.337962 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:00:50.337962 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:00:50.337962 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:00:50.337962 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:00:50.337962 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:00:50.337962 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:00:50.337962 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:00:50.337962 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:00:50.337962 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:00:50.337962 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:00:50.337962 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:00:50.337962 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:00:50.337962 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:00:50.338775 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:00:50.338775 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:00:50.338775 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:00:50.338775 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:00:50.338775 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:00:50.338775 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:00:50.338775 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:00:50.338775 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:00:50.338775 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:00:50.338775 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:00:50.338775 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:00:50.338775 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:00:50.338775 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 36/75] 10.222.0.98 () {56 vars in 1096 bytes} [Thu Aug 18 12:00:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:00:50.339480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06493839-82d3-49d7-9d9a-0a58e8b6f7f3 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:50.366360 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:50.371940 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb94121-8143-4335-a84a-e8fb01b22f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:50.372567 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb94121-8143-4335-a84a-e8fb01b22f12 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:50.372856 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb94121-8143-4335-a84a-e8fb01b22f12 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:50.373139 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb94121-8143-4335-a84a-e8fb01b22f12 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:50.373877 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb94121-8143-4335-a84a-e8fb01b22f12 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:00:50.374194 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb94121-8143-4335-a84a-e8fb01b22f12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:50.375662 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb94121-8143-4335-a84a-e8fb01b22f12 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:50.383869 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 40/76] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:00:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:50.384684 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb94121-8143-4335-a84a-e8fb01b22f12 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:50.442637 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06493839-82d3-49d7-9d9a-0a58e8b6f7f3 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:50.476866 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f4c35a-170e-4aa3-944a-9b76c755eb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:50.477497 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4c35a-170e-4aa3-944a-9b76c755eb95 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:50.477787 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4c35a-170e-4aa3-944a-9b76c755eb95 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:50.478071 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4c35a-170e-4aa3-944a-9b76c755eb95 None admin] PATH_INFO: `/v3/users/31feea8cb0e3454194acf739392519e2` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:50.487371 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb94121-8143-4335-a84a-e8fb01b22f12 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:50.492686 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f4c35a-170e-4aa3-944a-9b76c755eb95 None admin] RBAC: Authorizing `identity:get_user(user_id=31feea8cb0e3454194acf739392519e2)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:50.494192 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f4c35a-170e-4aa3-944a-9b76c755eb95 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:50.494321 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:50.496469 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 37/77] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:00:50 2022] GET /identity/v3/users/31feea8cb0e3454194acf739392519e2 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:50.498600 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f4c35a-170e-4aa3-944a-9b76c755eb95 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:50.524129 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6fdd97-f388-49e6-8574-3bf448535c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:50.524762 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fdd97-f388-49e6-8574-3bf448535c18 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:50.525049 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fdd97-f388-49e6-8574-3bf448535c18 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:50.525327 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fdd97-f388-49e6-8574-3bf448535c18 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:50.536420 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6fdd97-f388-49e6-8574-3bf448535c18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:00:50.541708 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-6c6fdd97-f388-49e6-8574-3bf448535c18 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 12:00:50.541708 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:00:50.541708 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:00:50.541708 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:00:50.541708 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:00:50.541708 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:00:50.541708 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:00:50.541708 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:00:50.541708 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:00:50.541708 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:00:50.541708 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:00:50.541708 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:00:50.541708 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:00:50.541708 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:00:50.541708 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:00:50.541708 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:00:50.543088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:00:50.543088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:00:50.543088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:00:50.543088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:00:50.543088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:00:50.543088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:00:50.543088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:00:50.543088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:00:50.543088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:00:50.543088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:00:50.543088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:00:50.543088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:00:50.543088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:00:50.543088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:00:50.543088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:00:50.543088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:00:50.548273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:00:50.548273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:00:50.548273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:00:50.548273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:00:50.548273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:00:50.548273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:00:50.548273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:00:50.548273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:00:50.548273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:00:50.548273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:00:50.548273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 12:00:50.548273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:00:50.548273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 41/78] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:00:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:00:50.548273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6fdd97-f388-49e6-8574-3bf448535c18 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:50.585674 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:50.601487 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f4c35a-170e-4aa3-944a-9b76c755eb95 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:50.636119 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0227a-94d8-4d97-8fbf-f3c34cf8bcc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:50.636755 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0227a-94d8-4d97-8fbf-f3c34cf8bcc0 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:50.637044 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0227a-94d8-4d97-8fbf-f3c34cf8bcc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:50.637328 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0227a-94d8-4d97-8fbf-f3c34cf8bcc0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:50.638468 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c0227a-94d8-4d97-8fbf-f3c34cf8bcc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:50.639979 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c0227a-94d8-4d97-8fbf-f3c34cf8bcc0 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:50.641549 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 38/79] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:00:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:50.642318 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0227a-94d8-4d97-8fbf-f3c34cf8bcc0 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:50.645024 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6fdd97-f388-49e6-8574-3bf448535c18 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:50.681288 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db481c68-36f4-486c-a138-c9df5d8369c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:50.681909 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db481c68-36f4-486c-a138-c9df5d8369c7 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:50.682199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db481c68-36f4-486c-a138-c9df5d8369c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:50.682505 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db481c68-36f4-486c-a138-c9df5d8369c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:50.683210 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db481c68-36f4-486c-a138-c9df5d8369c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:00:50.683552 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db481c68-36f4-486c-a138-c9df5d8369c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:50.684954 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db481c68-36f4-486c-a138-c9df5d8369c7 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:50.693766 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 42/80] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:00:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:50.694637 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db481c68-36f4-486c-a138-c9df5d8369c7 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:50.723824 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:50.755407 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0227a-94d8-4d97-8fbf-f3c34cf8bcc0 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:50.791493 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d1dd7-1d80-403a-a5f2-94eed52b0ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:50.792172 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d1dd7-1d80-403a-a5f2-94eed52b0ead None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:50.792460 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d1dd7-1d80-403a-a5f2-94eed52b0ead None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:50.792733 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d1dd7-1d80-403a-a5f2-94eed52b0ead None admin] PATH_INFO: `/v3/users/31feea8cb0e3454194acf739392519e2` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:50.793906 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169d1dd7-1d80-403a-a5f2-94eed52b0ead None admin] RBAC: Authorizing `identity:get_user(user_id=31feea8cb0e3454194acf739392519e2)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:50.795344 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169d1dd7-1d80-403a-a5f2-94eed52b0ead None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:50.796035 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db481c68-36f4-486c-a138-c9df5d8369c7 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:50.796943 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 39/81] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:00:50 2022] GET /identity/v3/users/31feea8cb0e3454194acf739392519e2 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:50.797720 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d1dd7-1d80-403a-a5f2-94eed52b0ead None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:50.819487 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:50.841163 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a7e347-0ed1-4db1-a0a8-7705b7e453e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:50.842038 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7e347-0ed1-4db1-a0a8-7705b7e453e1 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:50.842502 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7e347-0ed1-4db1-a0a8-7705b7e453e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:50.842957 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7e347-0ed1-4db1-a0a8-7705b7e453e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:50.920525 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d1dd7-1d80-403a-a5f2-94eed52b0ead None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:50.925628 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a7e347-0ed1-4db1-a0a8-7705b7e453e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:50.927108 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a7e347-0ed1-4db1-a0a8-7705b7e453e1 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:50.954371 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:50.957446 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1f32df-168b-4107-8e5e-307f1ebf777c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:50.958168 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1f32df-168b-4107-8e5e-307f1ebf777c None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:50.958530 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1f32df-168b-4107-8e5e-307f1ebf777c None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:50.958838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1f32df-168b-4107-8e5e-307f1ebf777c None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:50.959672 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1f32df-168b-4107-8e5e-307f1ebf777c None admin] RBAC: Adding query filter params (name=member) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:00:50.959982 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1f32df-168b-4107-8e5e-307f1ebf777c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:50.967549 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1f32df-168b-4107-8e5e-307f1ebf777c None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:50.970632 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 40/82] 10.222.0.98 () {56 vars in 1101 bytes} [Thu Aug 18 12:00:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:50.972369 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1f32df-168b-4107-8e5e-307f1ebf777c None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:51.027851 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 43/83] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:00:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:00:51.028714 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a7e347-0ed1-4db1-a0a8-7705b7e453e1 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:51.054795 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:51.072549 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1f32df-168b-4107-8e5e-307f1ebf777c None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:51.109810 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da6ca36-cd93-44d6-89c9-cb53174c1189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:51.110616 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6ca36-cd93-44d6-89c9-cb53174c1189 None admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:51.110616 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6ca36-cd93-44d6-89c9-cb53174c1189 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:51.110902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6ca36-cd93-44d6-89c9-cb53174c1189 None admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:51.111591 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da6ca36-cd93-44d6-89c9-cb53174c1189 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:51.113074 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da6ca36-cd93-44d6-89c9-cb53174c1189 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:51.133342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a7e347-0ed1-4db1-a0a8-7705b7e453e1 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:51.161101 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:51.168841 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a625ac3-08a2-4917-9347-f28f5f349185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:51.169483 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a625ac3-08a2-4917-9347-f28f5f349185 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:51.169771 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a625ac3-08a2-4917-9347-f28f5f349185 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:51.170057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a625ac3-08a2-4917-9347-f28f5f349185 None admin] PATH_INFO: `/v3/users/6c92bf2b004241b3b54d40ae3aaaefe8` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:51.171565 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a625ac3-08a2-4917-9347-f28f5f349185 None admin] RBAC: Authorizing `identity:get_user(user_id=6c92bf2b004241b3b54d40ae3aaaefe8)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:51.173031 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a625ac3-08a2-4917-9347-f28f5f349185 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:51.174622 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 44/84] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:00:51 2022] GET /identity/v3/users/6c92bf2b004241b3b54d40ae3aaaefe8 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:51.175508 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a625ac3-08a2-4917-9347-f28f5f349185 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:51.224579 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 41/85] 10.222.0.98 () {60 vars in 1132 bytes} [Thu Aug 18 12:00:50 2022] POST /identity/v3/users => generated 289 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:00:51.225424 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da6ca36-cd93-44d6-89c9-cb53174c1189 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:51.288744 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a625ac3-08a2-4917-9347-f28f5f349185 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:51.329493 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83f063-655d-4d67-b61a-6450377f6bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:51.330166 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83f063-655d-4d67-b61a-6450377f6bcd None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:51.330509 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83f063-655d-4d67-b61a-6450377f6bcd None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:51.330893 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83f063-655d-4d67-b61a-6450377f6bcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:51.378461 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da6ca36-cd93-44d6-89c9-cb53174c1189 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:51.400845 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:51.409885 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e83f063-655d-4d67-b61a-6450377f6bcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:51.411404 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e83f063-655d-4d67-b61a-6450377f6bcd None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:51.422281 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e949d-82d1-4cce-8683-c25b02fd4118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:51.423022 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e949d-82d1-4cce-8683-c25b02fd4118 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:51.423379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e949d-82d1-4cce-8683-c25b02fd4118 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:51.424008 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e949d-82d1-4cce-8683-c25b02fd4118 None admin] PATH_INFO: `/v3/users/31feea8cb0e3454194acf739392519e2` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:51.425205 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e949d-82d1-4cce-8683-c25b02fd4118 None admin] RBAC: Authorizing `identity:get_user(user_id=31feea8cb0e3454194acf739392519e2)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:51.426678 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e949d-82d1-4cce-8683-c25b02fd4118 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:51.428693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 42/86] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:00:51 2022] GET /identity/v3/users/31feea8cb0e3454194acf739392519e2 => generated 275 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:51.429107 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e949d-82d1-4cce-8683-c25b02fd4118 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:51.440106 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:51.553288 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 45/87] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:00:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:00:51.554011 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83f063-655d-4d67-b61a-6450377f6bcd None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:51.575691 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e949d-82d1-4cce-8683-c25b02fd4118 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:51.616094 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ed1301-e286-4127-95d0-2f58d38aea90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:51.616753 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed1301-e286-4127-95d0-2f58d38aea90 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:51.617041 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed1301-e286-4127-95d0-2f58d38aea90 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:51.617325 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed1301-e286-4127-95d0-2f58d38aea90 None admin] PATH_INFO: `/v3/projects/dede24cb1b014e9796b44d6cdac3ed69` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:51.618493 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ed1301-e286-4127-95d0-2f58d38aea90 None admin] RBAC: Authorizing `identity:get_project(project_id=dede24cb1b014e9796b44d6cdac3ed69)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:51.621338 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ed1301-e286-4127-95d0-2f58d38aea90 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:51.623833 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 43/88] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:00:51 2022] GET /identity/v3/projects/dede24cb1b014e9796b44d6cdac3ed69 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:51.624534 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ed1301-e286-4127-95d0-2f58d38aea90 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:51.685783 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:51.693864 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83f063-655d-4d67-b61a-6450377f6bcd None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:51.720747 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:51.742482 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d097d726-e9d4-4167-bf6a-2be7a80e588d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:51.743330 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097d726-e9d4-4167-bf6a-2be7a80e588d None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:51.743951 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097d726-e9d4-4167-bf6a-2be7a80e588d None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:51.744238 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097d726-e9d4-4167-bf6a-2be7a80e588d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:51.751405 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ed1301-e286-4127-95d0-2f58d38aea90 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:51.796516 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e584a6-772a-4d7d-8439-c1789d15618e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:51.797245 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e584a6-772a-4d7d-8439-c1789d15618e None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:51.797608 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e584a6-772a-4d7d-8439-c1789d15618e None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:51.797959 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e584a6-772a-4d7d-8439-c1789d15618e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:51.842280 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d097d726-e9d4-4167-bf6a-2be7a80e588d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:51.843794 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d097d726-e9d4-4167-bf6a-2be7a80e588d None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:51.889207 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e584a6-772a-4d7d-8439-c1789d15618e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:51.890874 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e584a6-772a-4d7d-8439-c1789d15618e None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:51.972968 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:51.979994 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:51.988159 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 46/89] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:00:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:00:51.992360 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d097d726-e9d4-4167-bf6a-2be7a80e588d None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:52.009328 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 44/90] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:00:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:00:52.010406 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e584a6-772a-4d7d-8439-c1789d15618e None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:52.123120 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e584a6-772a-4d7d-8439-c1789d15618e None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:52.129120 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d097d726-e9d4-4167-bf6a-2be7a80e588d None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:52.160898 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29255106-28e1-4a7b-abd2-2ff412e02898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:52.161730 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29255106-28e1-4a7b-abd2-2ff412e02898 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:52.162235 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29255106-28e1-4a7b-abd2-2ff412e02898 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:52.162657 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29255106-28e1-4a7b-abd2-2ff412e02898 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:52.163557 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29255106-28e1-4a7b-abd2-2ff412e02898 None admin] RBAC: Adding query filter params (role.id=c81e28908b9b49a38d33978df4d11716, scope.project.id=dede24cb1b014e9796b44d6cdac3ed69, user.id=31feea8cb0e3454194acf739392519e2) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:00:52.164010 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29255106-28e1-4a7b-abd2-2ff412e02898 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:52.165604 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29255106-28e1-4a7b-abd2-2ff412e02898 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:52.172293 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 45/91] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:00:52 2022] GET /identity/v3/role_assignments?user.id=31feea8cb0e3454194acf739392519e2&scope.project.id=dede24cb1b014e9796b44d6cdac3ed69&role.id=c81e28908b9b49a38d33978df4d11716 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:52.173258 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29255106-28e1-4a7b-abd2-2ff412e02898 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:52.184055 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca05aa-398f-495d-8a7b-0784a20e9b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:52.189059 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca05aa-398f-495d-8a7b-0784a20e9b1d None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:52.189621 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca05aa-398f-495d-8a7b-0784a20e9b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:52.190051 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca05aa-398f-495d-8a7b-0784a20e9b1d None admin] PATH_INFO: `/v3/projects/dede24cb1b014e9796b44d6cdac3ed69` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:52.191650 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca05aa-398f-495d-8a7b-0784a20e9b1d None admin] RBAC: Authorizing `identity:get_project(project_id=dede24cb1b014e9796b44d6cdac3ed69)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:52.193514 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca05aa-398f-495d-8a7b-0784a20e9b1d None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:52.199068 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 47/92] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:00:51 2022] GET /identity/v3/projects/dede24cb1b014e9796b44d6cdac3ed69 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:52.201434 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca05aa-398f-495d-8a7b-0784a20e9b1d None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:52.257622 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:52.338898 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29255106-28e1-4a7b-abd2-2ff412e02898 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:52.345982 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:52.388944 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c5580d-13b1-4e7d-acb3-867739a2e246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:52.390146 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5580d-13b1-4e7d-acb3-867739a2e246 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:52.390600 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5580d-13b1-4e7d-acb3-867739a2e246 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:52.391021 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5580d-13b1-4e7d-acb3-867739a2e246 None admin] PATH_INFO: `/v3/projects/dede24cb1b014e9796b44d6cdac3ed69` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:52.392372 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c5580d-13b1-4e7d-acb3-867739a2e246 None admin] RBAC: Authorizing `identity:get_project(project_id=dede24cb1b014e9796b44d6cdac3ed69)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:52.399503 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c5580d-13b1-4e7d-acb3-867739a2e246 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:52.399644 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca05aa-398f-495d-8a7b-0784a20e9b1d None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:52.401728 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 46/93] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:00:52 2022] GET /identity/v3/projects/dede24cb1b014e9796b44d6cdac3ed69 => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:52.402557 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c5580d-13b1-4e7d-acb3-867739a2e246 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:52.454159 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3535e-2a52-487b-bf8d-48f09624a4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:52.454814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3535e-2a52-487b-bf8d-48f09624a4ef None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:52.455103 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3535e-2a52-487b-bf8d-48f09624a4ef None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:52.455382 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3535e-2a52-487b-bf8d-48f09624a4ef None admin] PATH_INFO: `/v3/projects/10871d2fec9e407298cba25d8c3c6a3c` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:52.459244 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c3535e-2a52-487b-bf8d-48f09624a4ef None admin] RBAC: Authorizing `identity:get_project(project_id=10871d2fec9e407298cba25d8c3c6a3c)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:52.460976 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c3535e-2a52-487b-bf8d-48f09624a4ef None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:52.464960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 48/94] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:00:52 2022] GET /identity/v3/projects/10871d2fec9e407298cba25d8c3c6a3c => generated 312 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:52.465783 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3535e-2a52-487b-bf8d-48f09624a4ef None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:52.597227 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3535e-2a52-487b-bf8d-48f09624a4ef None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:52.633554 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab783e8e-e80b-4d84-94f8-4934041368f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:52.634171 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab783e8e-e80b-4d84-94f8-4934041368f2 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:52.634459 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab783e8e-e80b-4d84-94f8-4934041368f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:52.634735 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab783e8e-e80b-4d84-94f8-4934041368f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:52.635503 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab783e8e-e80b-4d84-94f8-4934041368f2 None admin] RBAC: Adding query filter params (role.id=c53a0e13b58642f291c856efe1f0b87d, scope.project.id=dede24cb1b014e9796b44d6cdac3ed69, user.id=6c92bf2b004241b3b54d40ae3aaaefe8) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:00:52.635821 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab783e8e-e80b-4d84-94f8-4934041368f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:52.636923 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:52.637824 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c5580d-13b1-4e7d-acb3-867739a2e246 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:52.643505 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab783e8e-e80b-4d84-94f8-4934041368f2 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:52.655614 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 49/95] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:00:52 2022] GET /identity/v3/role_assignments?user.id=6c92bf2b004241b3b54d40ae3aaaefe8&scope.project.id=dede24cb1b014e9796b44d6cdac3ed69&role.id=c53a0e13b58642f291c856efe1f0b87d => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:52.655614 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab783e8e-e80b-4d84-94f8-4934041368f2 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:52.656085 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:52.719614 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d36606-1c4d-45c2-a2ca-c0b715a0df25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:52.720247 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d36606-1c4d-45c2-a2ca-c0b715a0df25 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:52.720537 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d36606-1c4d-45c2-a2ca-c0b715a0df25 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:52.720820 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d36606-1c4d-45c2-a2ca-c0b715a0df25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:52.721566 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d36606-1c4d-45c2-a2ca-c0b715a0df25 None admin] RBAC: Adding query filter params (role.id=3a4b7e53b8644e508ac440f71138be16, scope.project.id=dede24cb1b014e9796b44d6cdac3ed69, user.id=31feea8cb0e3454194acf739392519e2) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:00:52.721884 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d36606-1c4d-45c2-a2ca-c0b715a0df25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:52.731821 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d36606-1c4d-45c2-a2ca-c0b715a0df25 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:52.739201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 47/96] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:00:52 2022] GET /identity/v3/role_assignments?user.id=31feea8cb0e3454194acf739392519e2&scope.project.id=dede24cb1b014e9796b44d6cdac3ed69&role.id=3a4b7e53b8644e508ac440f71138be16 => generated 259 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:52.774152 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab783e8e-e80b-4d84-94f8-4934041368f2 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:52.812908 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620094ed-2a7e-44d6-ab0a-46d6417fbef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:52.813540 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620094ed-2a7e-44d6-ab0a-46d6417fbef5 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:52.813827 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620094ed-2a7e-44d6-ab0a-46d6417fbef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:52.814106 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620094ed-2a7e-44d6-ab0a-46d6417fbef5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:52.814868 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620094ed-2a7e-44d6-ab0a-46d6417fbef5 None admin] RBAC: Adding query filter params (role.id=c81e28908b9b49a38d33978df4d11716, scope.project.id=10871d2fec9e407298cba25d8c3c6a3c, user.id=31feea8cb0e3454194acf739392519e2) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:00:52.815181 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620094ed-2a7e-44d6-ab0a-46d6417fbef5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:52.816752 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620094ed-2a7e-44d6-ab0a-46d6417fbef5 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:52.823543 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 50/97] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:00:52 2022] GET /identity/v3/role_assignments?user.id=31feea8cb0e3454194acf739392519e2&scope.project.id=10871d2fec9e407298cba25d8c3c6a3c&role.id=c81e28908b9b49a38d33978df4d11716 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:53.665473 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916c8b4-e576-4ebf-ac2f-d75bf69d8927 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:53.665991 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916c8b4-e576-4ebf-ac2f-d75bf69d8927 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:53.666071 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916c8b4-e576-4ebf-ac2f-d75bf69d8927 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:53.668193 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 48/98] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:00:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:00:53.673688 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277a245-c39f-4b10-b75c-25161ae3f926 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:53.674180 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277a245-c39f-4b10-b75c-25161ae3f926 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:53.674269 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277a245-c39f-4b10-b75c-25161ae3f926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:53.697487 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-8277a245-c39f-4b10-b75c-25161ae3f926 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:00:53.698741 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-8277a245-c39f-4b10-b75c-25161ae3f926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:00:53.771783 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:53.787562 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 51/99] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:00:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:00:54.141344 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40fcb8e-e14d-4175-8b13-e58ef4e33b6c None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:54.141636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40fcb8e-e14d-4175-8b13-e58ef4e33b6c None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:54.141923 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40fcb8e-e14d-4175-8b13-e58ef4e33b6c None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:54.143269 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 49/100] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:00:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:00:54.148868 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c03079-a3a2-49b4-8c6f-60b76e618da5 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:54.149165 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c03079-a3a2-49b4-8c6f-60b76e618da5 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:54.149451 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c03079-a3a2-49b4-8c6f-60b76e618da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:54.168820 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-37c03079-a3a2-49b4-8c6f-60b76e618da5 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:00:54.170097 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-37c03079-a3a2-49b4-8c6f-60b76e618da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:00:54.246472 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 52/101] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:00:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:00:54.335889 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40fcb8e-e14d-4175-8b13-e58ef4e33b6c None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:54.469629 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40fcb8e-e14d-4175-8b13-e58ef4e33b6c None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:54.514541 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692fea5-f14e-4c6b-a302-8a3fc279f6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:54.515207 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692fea5-f14e-4c6b-a302-8a3fc279f6f5 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:54.515530 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692fea5-f14e-4c6b-a302-8a3fc279f6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:54.515806 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692fea5-f14e-4c6b-a302-8a3fc279f6f5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:54.534458 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7692fea5-f14e-4c6b-a302-8a3fc279f6f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:00:54.539788 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-7692fea5-f14e-4c6b-a302-8a3fc279f6f5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 12:00:54.539788 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:00:54.539788 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:00:54.539788 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:00:54.539788 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:00:54.539788 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:00:54.539788 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:00:54.539788 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:00:54.539788 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:00:54.539788 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:00:54.539788 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:00:54.539788 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:00:54.539788 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:00:54.539788 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:00:54.539788 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:00:54.539788 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:00:54.540623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:00:54.540623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:00:54.540623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:00:54.540623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:00:54.540623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:00:54.540623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:00:54.540623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:00:54.540623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:00:54.540623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:00:54.540623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:00:54.540623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:00:54.540623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:00:54.540623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:00:54.540623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:00:54.540623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:00:54.540623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:00:54.541373 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:00:54.541373 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:00:54.541373 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:00:54.541373 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:00:54.541373 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:00:54.541373 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:00:54.541373 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:00:54.541373 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:00:54.541373 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:00:54.541373 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:00:54.541373 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 12:00:54.541373 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:00:54.541373 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 50/102] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:00:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:00:54.548209 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c03079-a3a2-49b4-8c6f-60b76e618da5 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:54.676944 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c03079-a3a2-49b4-8c6f-60b76e618da5 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:54.706792 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:54.726905 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9e340-2659-4cf8-9cd1-9783ee2d0a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:54.726905 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9e340-2659-4cf8-9cd1-9783ee2d0a08 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:54.726905 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9e340-2659-4cf8-9cd1-9783ee2d0a08 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:54.726905 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9e340-2659-4cf8-9cd1-9783ee2d0a08 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:54.727227 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9e340-2659-4cf8-9cd1-9783ee2d0a08 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:00:54.727227 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9e340-2659-4cf8-9cd1-9783ee2d0a08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:54.727613 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9e340-2659-4cf8-9cd1-9783ee2d0a08 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:54.739170 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 53/103] 10.222.0.98 () {56 vars in 1101 bytes} [Thu Aug 18 12:00:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:54.744514 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692fea5-f14e-4c6b-a302-8a3fc279f6f5 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:54.776676 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:54.793009 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9e340-2659-4cf8-9cd1-9783ee2d0a08 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:54.874784 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692fea5-f14e-4c6b-a302-8a3fc279f6f5 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:54.938733 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e49a401-067e-4c19-9ceb-23642ab99e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:54.939396 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49a401-067e-4c19-9ceb-23642ab99e92 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:54.939712 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49a401-067e-4c19-9ceb-23642ab99e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:54.940284 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49a401-067e-4c19-9ceb-23642ab99e92 None admin] PATH_INFO: `/v3/users/955131e7b5f048a5be55cc222561df18` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:54.966033 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e49a401-067e-4c19-9ceb-23642ab99e92 None admin] RBAC: Authorizing `identity:get_user(user_id=955131e7b5f048a5be55cc222561df18)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:54.967822 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e49a401-067e-4c19-9ceb-23642ab99e92 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:54.970728 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 51/104] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:00:54 2022] GET /identity/v3/users/955131e7b5f048a5be55cc222561df18 => generated 289 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:54.976772 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9e340-2659-4cf8-9cd1-9783ee2d0a08 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:54.977885 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e49a401-067e-4c19-9ceb-23642ab99e92 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:55.052562 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41094b6-11cf-4a44-8c78-565e2666dccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:55.053508 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41094b6-11cf-4a44-8c78-565e2666dccb None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:55.053973 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41094b6-11cf-4a44-8c78-565e2666dccb None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:55.054427 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41094b6-11cf-4a44-8c78-565e2666dccb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:55.055809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41094b6-11cf-4a44-8c78-565e2666dccb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:55.057443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41094b6-11cf-4a44-8c78-565e2666dccb None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:55.059808 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 54/105] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:00:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:00:55.068388 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41094b6-11cf-4a44-8c78-565e2666dccb None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:55.178636 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:55.202982 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:55.222569 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e49a401-067e-4c19-9ceb-23642ab99e92 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:55.282446 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41094b6-11cf-4a44-8c78-565e2666dccb None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:55.287783 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9302578e-ae57-44fd-9e02-1957737b402c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:55.288710 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9302578e-ae57-44fd-9e02-1957737b402c None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:55.289272 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9302578e-ae57-44fd-9e02-1957737b402c None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:55.290402 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9302578e-ae57-44fd-9e02-1957737b402c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:55.320207 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b684b-b813-4180-b9eb-20d90c429707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:55.321083 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b684b-b813-4180-b9eb-20d90c429707 None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:55.321539 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b684b-b813-4180-b9eb-20d90c429707 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:55.322003 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b684b-b813-4180-b9eb-20d90c429707 None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:55.322880 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b684b-b813-4180-b9eb-20d90c429707 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:55.325563 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b684b-b813-4180-b9eb-20d90c429707 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:55.370182 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9302578e-ae57-44fd-9e02-1957737b402c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:55.372207 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9302578e-ae57-44fd-9e02-1957737b402c None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:55.412088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 55/106] 10.222.0.98 () {60 vars in 1141 bytes} [Thu Aug 18 12:00:55 2022] POST /identity/v3/projects => generated 302 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:00:55.414942 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ec064-e50b-4489-9582-f10b99dad823 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:55.415491 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ec064-e50b-4489-9582-f10b99dad823 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:55.415990 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ec064-e50b-4489-9582-f10b99dad823 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:55.419595 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 56/107] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:00:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:00:55.421292 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742aab9f-e628-4a99-be18-f68083a462ee None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:55.421780 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742aab9f-e628-4a99-be18-f68083a462ee None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:55.422231 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742aab9f-e628-4a99-be18-f68083a462ee None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:55.429264 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 57/108] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:00:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:00:55.429264 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c5f19-1086-4e51-9c53-fa1b6f454fe5 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:55.429264 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c5f19-1086-4e51-9c53-fa1b6f454fe5 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:55.429553 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c5f19-1086-4e51-9c53-fa1b6f454fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:55.474086 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-ce7c5f19-1086-4e51-9c53-fa1b6f454fe5 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:00:55.475343 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-ce7c5f19-1086-4e51-9c53-fa1b6f454fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:00:55.508392 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:55.522833 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 52/109] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:00:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:00:55.524235 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886fa36e-e906-4a2e-ad0a-844fa80a3379 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:55.524498 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886fa36e-e906-4a2e-ad0a-844fa80a3379 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:55.524783 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886fa36e-e906-4a2e-ad0a-844fa80a3379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:55.555213 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-886fa36e-e906-4a2e-ad0a-844fa80a3379 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:00:55.556411 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-886fa36e-e906-4a2e-ad0a-844fa80a3379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:00:55.563310 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 58/110] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:00:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:00:55.565153 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e82697-1167-48e5-a439-515888305ff7 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:55.565642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e82697-1167-48e5-a439-515888305ff7 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:55.566138 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e82697-1167-48e5-a439-515888305ff7 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:55.568065 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 59/111] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:00:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:00:55.569032 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e82697-1167-48e5-a439-515888305ff7 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:55.610428 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:55.639478 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 53/112] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:00:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:00:55.640844 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe5f605-f9d3-4875-a382-d488aa7e232f None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:55.641138 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe5f605-f9d3-4875-a382-d488aa7e232f None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:55.641427 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe5f605-f9d3-4875-a382-d488aa7e232f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:55.684698 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-6fe5f605-f9d3-4875-a382-d488aa7e232f None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:00:55.685928 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-6fe5f605-f9d3-4875-a382-d488aa7e232f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:00:55.696948 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e82697-1167-48e5-a439-515888305ff7 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:55.735742 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dfe4ae-33c4-4a05-a93b-31eb9ca09022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:55.736519 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dfe4ae-33c4-4a05-a93b-31eb9ca09022 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:55.736871 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dfe4ae-33c4-4a05-a93b-31eb9ca09022 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:55.737213 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dfe4ae-33c4-4a05-a93b-31eb9ca09022 None admin] PATH_INFO: `/v3/projects/dede24cb1b014e9796b44d6cdac3ed69` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:55.738397 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dfe4ae-33c4-4a05-a93b-31eb9ca09022 None admin] RBAC: Authorizing `identity:get_project(project_id=dede24cb1b014e9796b44d6cdac3ed69)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:55.740184 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dfe4ae-33c4-4a05-a93b-31eb9ca09022 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:55.746920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 60/113] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:00:55 2022] GET /identity/v3/projects/dede24cb1b014e9796b44d6cdac3ed69 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:55.748329 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d3bc4f-16f6-4be8-9966-c0610089e3e5 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:55.748689 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d3bc4f-16f6-4be8-9966-c0610089e3e5 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:55.749038 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d3bc4f-16f6-4be8-9966-c0610089e3e5 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:55.754651 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 61/114] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:00:55.754651 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d3bc4f-16f6-4be8-9966-c0610089e3e5 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:55.788867 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 54/115] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:00:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:00:55.790915 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f6384-f28b-4023-a788-5a438e2a49f2 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:55.791357 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f6384-f28b-4023-a788-5a438e2a49f2 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:55.791918 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f6384-f28b-4023-a788-5a438e2a49f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:55.812912 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-773f6384-f28b-4023-a788-5a438e2a49f2 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:00:55.814397 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-773f6384-f28b-4023-a788-5a438e2a49f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:00:55.869696 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:55.873514 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:55.878688 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d3bc4f-16f6-4be8-9966-c0610089e3e5 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:55.898162 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 55/116] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:00:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:00:55.923319 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67822431-0cc6-4380-9be6-ceb2652869f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:55.923975 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67822431-0cc6-4380-9be6-ceb2652869f8 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:55.924249 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67822431-0cc6-4380-9be6-ceb2652869f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:55.924524 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67822431-0cc6-4380-9be6-ceb2652869f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:55.925259 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67822431-0cc6-4380-9be6-ceb2652869f8 None admin] RBAC: Adding query filter params (role.id=93cdab4d25f54b8ca825e7ee5a5dc2d2, scope.project.id=dede24cb1b014e9796b44d6cdac3ed69, user.id=955131e7b5f048a5be55cc222561df18) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:00:55.925571 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67822431-0cc6-4380-9be6-ceb2652869f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:55.927376 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f6384-f28b-4023-a788-5a438e2a49f2 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:55.927542 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67822431-0cc6-4380-9be6-ceb2652869f8 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:55.936184 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 62/117] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:00:55 2022] GET /identity/v3/role_assignments?user.id=955131e7b5f048a5be55cc222561df18&scope.project.id=dede24cb1b014e9796b44d6cdac3ed69&role.id=93cdab4d25f54b8ca825e7ee5a5dc2d2 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:00:56.051687 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f6384-f28b-4023-a788-5a438e2a49f2 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:56.093876 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b0ec3-0a55-4b8f-b57a-25fc9991aaf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:56.094901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b0ec3-0a55-4b8f-b57a-25fc9991aaf7 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:56.095378 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b0ec3-0a55-4b8f-b57a-25fc9991aaf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:56.095871 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b0ec3-0a55-4b8f-b57a-25fc9991aaf7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:56.108875 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b0ec3-0a55-4b8f-b57a-25fc9991aaf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:00:56.114056 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-688b0ec3-0a55-4b8f-b57a-25fc9991aaf7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:00:56.114056 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:00:56.114056 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:00:56.114056 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:00:56.114056 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:00:56.114056 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:00:56.114056 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:00:56.114056 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:00:56.114056 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:00:56.114056 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:00:56.114056 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:00:56.114056 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:00:56.114056 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:00:56.114056 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:00:56.114056 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:00:56.114056 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:00:56.114891 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:00:56.114891 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:00:56.114891 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:00:56.114891 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:00:56.114891 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:00:56.114891 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:00:56.114891 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:00:56.114891 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:00:56.114891 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:00:56.114891 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:00:56.114891 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:00:56.114891 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:00:56.114891 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:00:56.114891 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:00:56.114891 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:00:56.114891 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:00:56.115710 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:00:56.115710 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:00:56.115710 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:00:56.115710 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:00:56.115710 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:00:56.115710 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:00:56.115710 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:00:56.115710 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:00:56.115710 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:00:56.115710 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:00:56.115710 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:00:56.115710 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:00:56.117988 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 56/118] 10.222.0.98 () {56 vars in 1096 bytes} [Thu Aug 18 12:00:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:00:56.124815 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67822431-0cc6-4380-9be6-ceb2652869f8 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:56.224362 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b0ec3-0a55-4b8f-b57a-25fc9991aaf7 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:56.250874 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67822431-0cc6-4380-9be6-ceb2652869f8 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:56.305034 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2de100a-9201-483b-937e-c4cc069556de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:56.305664 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de100a-9201-483b-937e-c4cc069556de None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:56.305948 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de100a-9201-483b-937e-c4cc069556de None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:56.306242 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de100a-9201-483b-937e-c4cc069556de None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:56.306911 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2de100a-9201-483b-937e-c4cc069556de None admin] RBAC: Adding query filter params (name=admin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:00:56.307217 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2de100a-9201-483b-937e-c4cc069556de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:56.307782 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:56.308707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2de100a-9201-483b-937e-c4cc069556de None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:56.321827 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 63/119] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:00:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:56.322546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2de100a-9201-483b-937e-c4cc069556de None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:56.353432 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:56.395454 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b0ec3-0a55-4b8f-b57a-25fc9991aaf7 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:56.435370 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2de100a-9201-483b-937e-c4cc069556de None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:56.443381 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b86458-cf92-4dc5-9014-014b4a407556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:56.444496 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b86458-cf92-4dc5-9014-014b4a407556 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:56.444777 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b86458-cf92-4dc5-9014-014b4a407556 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:56.445056 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b86458-cf92-4dc5-9014-014b4a407556 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:56.459234 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b86458-cf92-4dc5-9014-014b4a407556 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:00:56.466103 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-a5b86458-cf92-4dc5-9014-014b4a407556 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 12:00:56.466103 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:00:56.466103 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:00:56.466103 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:00:56.466103 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:00:56.466103 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:00:56.466103 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:00:56.466103 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:00:56.466103 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:00:56.466103 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:00:56.466103 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:00:56.466103 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:00:56.466103 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:00:56.466103 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:00:56.466103 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:00:56.466103 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:00:56.466940 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:00:56.466940 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:00:56.466940 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:00:56.466940 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:00:56.466940 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:00:56.466940 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:00:56.466940 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:00:56.466940 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:00:56.466940 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:00:56.466940 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:00:56.466940 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:00:56.466940 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:00:56.466940 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:00:56.466940 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:00:56.466940 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:00:56.466940 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:00:56.467711 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:00:56.467711 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:00:56.467711 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:00:56.467711 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:00:56.467711 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:00:56.467711 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:00:56.467711 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:00:56.467711 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:00:56.467711 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:00:56.467711 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:00:56.467711 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 12:00:56.467711 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:00:56.467711 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 57/120] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:00:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:00:56.468763 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b86458-cf92-4dc5-9014-014b4a407556 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:56.474230 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86de9e1-9074-45f7-959a-f4ca46bf66f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:56.475009 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86de9e1-9074-45f7-959a-f4ca46bf66f1 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:56.475400 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86de9e1-9074-45f7-959a-f4ca46bf66f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:56.475809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86de9e1-9074-45f7-959a-f4ca46bf66f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:56.486924 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86de9e1-9074-45f7-959a-f4ca46bf66f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:00:56.492414 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-f86de9e1-9074-45f7-959a-f4ca46bf66f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 12:00:56.492414 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:00:56.492414 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:00:56.492414 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:00:56.492414 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:00:56.492414 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:00:56.492414 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:00:56.492414 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:00:56.492414 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:00:56.492414 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:00:56.492414 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:00:56.492414 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:00:56.492414 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:00:56.492414 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:00:56.492414 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:00:56.492414 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:00:56.494649 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:00:56.494649 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:00:56.494649 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:00:56.494649 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:00:56.494649 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:00:56.494649 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:00:56.494649 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:00:56.494649 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:00:56.494649 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:00:56.494649 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:00:56.494649 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:00:56.494649 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:00:56.494649 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:00:56.494649 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:00:56.494649 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:00:56.494649 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:00:56.496950 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:00:56.496950 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:00:56.496950 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:00:56.496950 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:00:56.496950 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:00:56.496950 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:00:56.496950 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:00:56.496950 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:00:56.496950 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:00:56.496950 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:00:56.496950 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 12:00:56.496950 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:00:56.496950 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 64/121] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:00:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:00:56.496950 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86de9e1-9074-45f7-959a-f4ca46bf66f1 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:56.591861 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:56.597823 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b86458-cf92-4dc5-9014-014b4a407556 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:56.610788 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:56.618333 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86de9e1-9074-45f7-959a-f4ca46bf66f1 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:56.642894 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb1bff-1f54-4332-b321-a1f8139f60cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:56.643861 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb1bff-1f54-4332-b321-a1f8139f60cb None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:56.644297 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb1bff-1f54-4332-b321-a1f8139f60cb None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:56.644588 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb1bff-1f54-4332-b321-a1f8139f60cb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:56.656122 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81281b1-0f98-4dfb-bf68-6d152641ff80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:56.657082 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81281b1-0f98-4dfb-bf68-6d152641ff80 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:56.657685 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81281b1-0f98-4dfb-bf68-6d152641ff80 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:56.657964 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fb1bff-1f54-4332-b321-a1f8139f60cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:00:56.658378 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81281b1-0f98-4dfb-bf68-6d152641ff80 None admin] PATH_INFO: `/v3/users/6c92bf2b004241b3b54d40ae3aaaefe8` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:56.659779 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81281b1-0f98-4dfb-bf68-6d152641ff80 None admin] RBAC: Authorizing `identity:get_user(user_id=6c92bf2b004241b3b54d40ae3aaaefe8)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:56.661617 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81281b1-0f98-4dfb-bf68-6d152641ff80 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:56.663451 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 65/122] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:00:56 2022] GET /identity/v3/users/6c92bf2b004241b3b54d40ae3aaaefe8 => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:56.664518 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81281b1-0f98-4dfb-bf68-6d152641ff80 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:56.668030 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-a2fb1bff-1f54-4332-b321-a1f8139f60cb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 12:00:56.668030 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:00:56.668030 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:00:56.668030 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:00:56.668030 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:00:56.668030 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:00:56.668030 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:00:56.668030 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:00:56.668030 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:00:56.668030 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:00:56.668030 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:00:56.668030 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:00:56.668030 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:00:56.668030 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:00:56.668030 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:00:56.668030 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:00:56.668882 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:00:56.668882 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:00:56.668882 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:00:56.668882 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:00:56.668882 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:00:56.668882 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:00:56.668882 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:00:56.668882 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:00:56.668882 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:00:56.668882 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:00:56.668882 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:00:56.668882 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:00:56.668882 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:00:56.668882 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:00:56.668882 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:00:56.668882 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:00:56.669644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:00:56.669644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:00:56.669644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:00:56.669644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:00:56.669644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:00:56.669644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:00:56.669644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:00:56.669644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:00:56.669644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:00:56.669644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:00:56.669644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 12:00:56.669644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:00:56.669644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 58/123] 10.222.0.98 () {56 vars in 1114 bytes} [Thu Aug 18 12:00:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:00:56.670306 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb1bff-1f54-4332-b321-a1f8139f60cb None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:56.788468 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81281b1-0f98-4dfb-bf68-6d152641ff80 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:56.812756 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb1bff-1f54-4332-b321-a1f8139f60cb None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:56.827020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd01c7da-79d9-481a-8aac-d956fd0c1283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:56.827942 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01c7da-79d9-481a-8aac-d956fd0c1283 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:56.828426 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01c7da-79d9-481a-8aac-d956fd0c1283 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:56.828897 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01c7da-79d9-481a-8aac-d956fd0c1283 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:56.829782 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd01c7da-79d9-481a-8aac-d956fd0c1283 None admin] RBAC: Adding query filter params (name=member) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:00:56.830365 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd01c7da-79d9-481a-8aac-d956fd0c1283 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:56.832018 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd01c7da-79d9-481a-8aac-d956fd0c1283 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:56.841655 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 66/124] 10.222.0.98 () {56 vars in 1101 bytes} [Thu Aug 18 12:00:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:56.842783 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd01c7da-79d9-481a-8aac-d956fd0c1283 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:56.854117 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856f9bca-aa05-4739-b17d-717abd112658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:56.854739 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f9bca-aa05-4739-b17d-717abd112658 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:56.855117 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f9bca-aa05-4739-b17d-717abd112658 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:56.855941 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f9bca-aa05-4739-b17d-717abd112658 None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:56.856643 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856f9bca-aa05-4739-b17d-717abd112658 None admin] RBAC: Adding query filter params (name=member) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:00:56.857414 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856f9bca-aa05-4739-b17d-717abd112658 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:56.859225 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856f9bca-aa05-4739-b17d-717abd112658 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:56.865748 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:56.871524 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 59/125] 10.222.0.98 () {56 vars in 1101 bytes} [Thu Aug 18 12:00:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:56.876439 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856f9bca-aa05-4739-b17d-717abd112658 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:56.900061 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:56.981199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd01c7da-79d9-481a-8aac-d956fd0c1283 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:56.997622 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856f9bca-aa05-4739-b17d-717abd112658 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:57.029146 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284f19e-0b59-4a7c-ae8d-8f4c22c56413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:57.030392 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284f19e-0b59-4a7c-ae8d-8f4c22c56413 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:57.031253 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284f19e-0b59-4a7c-ae8d-8f4c22c56413 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:57.032023 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284f19e-0b59-4a7c-ae8d-8f4c22c56413 None admin] PATH_INFO: `/v3/projects/dede24cb1b014e9796b44d6cdac3ed69` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:57.033402 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4284f19e-0b59-4a7c-ae8d-8f4c22c56413 None admin] RBAC: Authorizing `identity:get_project(project_id=dede24cb1b014e9796b44d6cdac3ed69)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:57.035083 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4284f19e-0b59-4a7c-ae8d-8f4c22c56413 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:57.035751 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e1924-0d93-46a8-aa1e-d9a3a4df088f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:57.036371 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e1924-0d93-46a8-aa1e-d9a3a4df088f None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:57.036660 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e1924-0d93-46a8-aa1e-d9a3a4df088f None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:57.036942 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e1924-0d93-46a8-aa1e-d9a3a4df088f None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:57.037617 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e1924-0d93-46a8-aa1e-d9a3a4df088f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:00:57.037932 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e1924-0d93-46a8-aa1e-d9a3a4df088f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:57.038221 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 67/126] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:00:56 2022] GET /identity/v3/projects/dede24cb1b014e9796b44d6cdac3ed69 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:57.043071 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e1924-0d93-46a8-aa1e-d9a3a4df088f None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:57.044644 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284f19e-0b59-4a7c-ae8d-8f4c22c56413 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:57.050360 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 60/127] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:00:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:57.051039 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e1924-0d93-46a8-aa1e-d9a3a4df088f None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:57.158625 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:57.162026 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:57.177401 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284f19e-0b59-4a7c-ae8d-8f4c22c56413 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:57.181248 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e1924-0d93-46a8-aa1e-d9a3a4df088f None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:57.217684 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747445d-c33c-4eb4-8069-a59bd3568b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:57.218411 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747445d-c33c-4eb4-8069-a59bd3568b70 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:57.218707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747445d-c33c-4eb4-8069-a59bd3568b70 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:57.218987 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747445d-c33c-4eb4-8069-a59bd3568b70 None admin] PATH_INFO: `/v3/users/31feea8cb0e3454194acf739392519e2` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:57.220263 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6747445d-c33c-4eb4-8069-a59bd3568b70 None admin] RBAC: Authorizing `identity:get_user(user_id=31feea8cb0e3454194acf739392519e2)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:57.221848 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be58199-072d-4075-8193-638b1e9045bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:57.221991 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6747445d-c33c-4eb4-8069-a59bd3568b70 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:57.222901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be58199-072d-4075-8193-638b1e9045bd None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:57.223992 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 68/128] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:00:57 2022] GET /identity/v3/users/31feea8cb0e3454194acf739392519e2 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:57.224809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747445d-c33c-4eb4-8069-a59bd3568b70 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:57.224994 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be58199-072d-4075-8193-638b1e9045bd None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:57.225477 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be58199-072d-4075-8193-638b1e9045bd None admin] PATH_INFO: `/v3/users/31feea8cb0e3454194acf739392519e2` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:57.226940 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be58199-072d-4075-8193-638b1e9045bd None admin] RBAC: Authorizing `identity:get_user(user_id=31feea8cb0e3454194acf739392519e2)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:57.229033 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be58199-072d-4075-8193-638b1e9045bd None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:57.235885 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 61/129] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:00:57 2022] GET /identity/v3/users/31feea8cb0e3454194acf739392519e2 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:57.237223 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be58199-072d-4075-8193-638b1e9045bd None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:57.349169 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747445d-c33c-4eb4-8069-a59bd3568b70 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:57.363016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be58199-072d-4075-8193-638b1e9045bd None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:57.390715 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e1ece-b1de-42c5-88c5-b09c24ae8bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:57.391772 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e1ece-b1de-42c5-88c5-b09c24ae8bfd None admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:57.392252 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e1ece-b1de-42c5-88c5-b09c24ae8bfd None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:57.392866 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e1ece-b1de-42c5-88c5-b09c24ae8bfd None admin] PATH_INFO: `/v3/projects/dede24cb1b014e9796b44d6cdac3ed69/users/6c92bf2b004241b3b54d40ae3aaaefe8/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:57.395321 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e1ece-b1de-42c5-88c5-b09c24ae8bfd None admin] RBAC: Authorizing `identity:create_grant(project_id=dede24cb1b014e9796b44d6cdac3ed69, user_id=6c92bf2b004241b3b54d40ae3aaaefe8, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:57.397201 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e1ece-b1de-42c5-88c5-b09c24ae8bfd None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:57.407815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f48f4d-df05-4c1e-a119-6c210ee0a39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:57.408458 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f48f4d-df05-4c1e-a119-6c210ee0a39f None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:57.408807 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f48f4d-df05-4c1e-a119-6c210ee0a39f None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:57.409023 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f48f4d-df05-4c1e-a119-6c210ee0a39f None admin] PATH_INFO: `/v3/users/31feea8cb0e3454194acf739392519e2` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:57.410157 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f48f4d-df05-4c1e-a119-6c210ee0a39f None admin] RBAC: Authorizing `identity:get_user(user_id=31feea8cb0e3454194acf739392519e2)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:57.411644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f48f4d-df05-4c1e-a119-6c210ee0a39f None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:57.413332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 62/130] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:00:57 2022] GET /identity/v3/users/31feea8cb0e3454194acf739392519e2 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:57.413977 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f48f4d-df05-4c1e-a119-6c210ee0a39f None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:57.436775 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:57.466997 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 69/131] 10.222.0.98 () {58 vars in 1439 bytes} [Thu Aug 18 12:00:57 2022] PUT /identity/v3/projects/dede24cb1b014e9796b44d6cdac3ed69/users/6c92bf2b004241b3b54d40ae3aaaefe8/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:00:57.468538 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e1ece-b1de-42c5-88c5-b09c24ae8bfd None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:57.491060 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:57.532459 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f48f4d-df05-4c1e-a119-6c210ee0a39f None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:57.573109 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73605722-b44b-4b41-9091-ebddb3e178ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:57.573953 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73605722-b44b-4b41-9091-ebddb3e178ed None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:57.574242 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73605722-b44b-4b41-9091-ebddb3e178ed None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:57.574522 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73605722-b44b-4b41-9091-ebddb3e178ed None admin] PATH_INFO: `/v3/projects/dede24cb1b014e9796b44d6cdac3ed69` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:57.576081 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73605722-b44b-4b41-9091-ebddb3e178ed None admin] RBAC: Authorizing `identity:get_project(project_id=dede24cb1b014e9796b44d6cdac3ed69)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:57.577467 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73605722-b44b-4b41-9091-ebddb3e178ed None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:57.579472 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 63/132] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:00:57 2022] GET /identity/v3/projects/dede24cb1b014e9796b44d6cdac3ed69 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:57.580495 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73605722-b44b-4b41-9091-ebddb3e178ed None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:57.641691 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e1ece-b1de-42c5-88c5-b09c24ae8bfd None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:57.717190 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67262b-3e97-4b47-a7bb-363bc2dcc461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:57.718214 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67262b-3e97-4b47-a7bb-363bc2dcc461 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:57.718696 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67262b-3e97-4b47-a7bb-363bc2dcc461 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:57.719186 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67262b-3e97-4b47-a7bb-363bc2dcc461 None admin] PATH_INFO: `/v3/projects/10871d2fec9e407298cba25d8c3c6a3c` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:57.720543 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a67262b-3e97-4b47-a7bb-363bc2dcc461 None admin] RBAC: Authorizing `identity:get_project(project_id=10871d2fec9e407298cba25d8c3c6a3c)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:57.722252 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a67262b-3e97-4b47-a7bb-363bc2dcc461 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:57.724107 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 70/133] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:00:57 2022] GET /identity/v3/projects/10871d2fec9e407298cba25d8c3c6a3c => generated 312 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:57.724989 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67262b-3e97-4b47-a7bb-363bc2dcc461 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:57.726991 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:57.746824 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73605722-b44b-4b41-9091-ebddb3e178ed None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:57.787698 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ed669-a573-45ac-aef4-299c81da1472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:57.788354 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ed669-a573-45ac-aef4-299c81da1472 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:57.788641 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ed669-a573-45ac-aef4-299c81da1472 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:57.788923 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ed669-a573-45ac-aef4-299c81da1472 None admin] PATH_INFO: `/v3/projects/dede24cb1b014e9796b44d6cdac3ed69` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:57.790236 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995ed669-a573-45ac-aef4-299c81da1472 None admin] RBAC: Authorizing `identity:get_project(project_id=dede24cb1b014e9796b44d6cdac3ed69)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:57.794198 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995ed669-a573-45ac-aef4-299c81da1472 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:57.796526 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 64/134] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:00:57 2022] GET /identity/v3/projects/dede24cb1b014e9796b44d6cdac3ed69 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:57.797369 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ed669-a573-45ac-aef4-299c81da1472 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:57.848090 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:57.875036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67262b-3e97-4b47-a7bb-363bc2dcc461 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:57.934636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ed669-a573-45ac-aef4-299c81da1472 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:57.940748 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea93792-20c0-4276-90f6-c619b3526f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:57.941780 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea93792-20c0-4276-90f6-c619b3526f63 None admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:57.942300 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea93792-20c0-4276-90f6-c619b3526f63 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:57.943337 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea93792-20c0-4276-90f6-c619b3526f63 None admin] PATH_INFO: `/v3/projects/dede24cb1b014e9796b44d6cdac3ed69/users/31feea8cb0e3454194acf739392519e2/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:57.946089 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea93792-20c0-4276-90f6-c619b3526f63 None admin] RBAC: Authorizing `identity:create_grant(project_id=dede24cb1b014e9796b44d6cdac3ed69, user_id=31feea8cb0e3454194acf739392519e2, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:57.948472 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea93792-20c0-4276-90f6-c619b3526f63 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:57.975057 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a832106-0eae-49e2-b767-022db6cc15b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:57.975916 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a832106-0eae-49e2-b767-022db6cc15b7 None admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:57.976279 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a832106-0eae-49e2-b767-022db6cc15b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:57.976627 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a832106-0eae-49e2-b767-022db6cc15b7 None admin] PATH_INFO: `/v3/projects/10871d2fec9e407298cba25d8c3c6a3c/users/31feea8cb0e3454194acf739392519e2/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:57.978797 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a832106-0eae-49e2-b767-022db6cc15b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=10871d2fec9e407298cba25d8c3c6a3c, user_id=31feea8cb0e3454194acf739392519e2, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:57.980347 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a832106-0eae-49e2-b767-022db6cc15b7 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:58.048182 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 71/135] 10.222.0.98 () {58 vars in 1439 bytes} [Thu Aug 18 12:00:57 2022] PUT /identity/v3/projects/dede24cb1b014e9796b44d6cdac3ed69/users/31feea8cb0e3454194acf739392519e2/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:00:58.049770 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea93792-20c0-4276-90f6-c619b3526f63 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:58.091716 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 65/136] 10.222.0.98 () {58 vars in 1439 bytes} [Thu Aug 18 12:00:57 2022] PUT /identity/v3/projects/10871d2fec9e407298cba25d8c3c6a3c/users/31feea8cb0e3454194acf739392519e2/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:00:58.096595 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef8887-4085-44bf-87ee-3f02962492b4 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:58.097064 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef8887-4085-44bf-87ee-3f02962492b4 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:58.097458 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef8887-4085-44bf-87ee-3f02962492b4 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:58.099217 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 66/137] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:00:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:00:58.108495 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a254da-ef82-4977-964b-c776ea43a445 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:58.108822 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a254da-ef82-4977-964b-c776ea43a445 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:58.109603 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a254da-ef82-4977-964b-c776ea43a445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:58.154163 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-93a254da-ef82-4977-964b-c776ea43a445 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:00:58.156033 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-93a254da-ef82-4977-964b-c776ea43a445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:00:58.179457 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:58.262486 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea93792-20c0-4276-90f6-c619b3526f63 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:58.289451 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 67/138] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:00:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:00:58.302263 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e269c-3693-4564-b20d-78577592fe8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:58.302690 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43e231-7816-4dbf-81ea-cf39122b3f4c None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:58.302956 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e269c-3693-4564-b20d-78577592fe8a None admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:58.303250 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e269c-3693-4564-b20d-78577592fe8a None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:58.303563 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e269c-3693-4564-b20d-78577592fe8a None admin] PATH_INFO: `/v3/projects/dede24cb1b014e9796b44d6cdac3ed69/users/31feea8cb0e3454194acf739392519e2/roles/3a4b7e53b8644e508ac440f71138be16` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:58.303810 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43e231-7816-4dbf-81ea-cf39122b3f4c None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:58.304273 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43e231-7816-4dbf-81ea-cf39122b3f4c None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:58.306048 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825e269c-3693-4564-b20d-78577592fe8a None admin] RBAC: Authorizing `identity:create_grant(project_id=dede24cb1b014e9796b44d6cdac3ed69, user_id=31feea8cb0e3454194acf739392519e2, role_id=3a4b7e53b8644e508ac440f71138be16)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:58.306424 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 68/139] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:00:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 12:00:58.307659 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825e269c-3693-4564-b20d-78577592fe8a None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:58.312607 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e13b84-b661-4a9a-b055-5a55155c2e25 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:58.313406 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e13b84-b661-4a9a-b055-5a55155c2e25 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:58.313534 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e13b84-b661-4a9a-b055-5a55155c2e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:58.338749 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-98e13b84-b661-4a9a-b055-5a55155c2e25 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:00:58.340381 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-98e13b84-b661-4a9a-b055-5a55155c2e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:00:58.410207 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:58.415341 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 72/140] 10.222.0.98 () {58 vars in 1439 bytes} [Thu Aug 18 12:00:58 2022] PUT /identity/v3/projects/dede24cb1b014e9796b44d6cdac3ed69/users/31feea8cb0e3454194acf739392519e2/roles/3a4b7e53b8644e508ac440f71138be16 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:00:58.437528 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 69/141] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:00:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 12:00:58.685457 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e269c-3693-4564-b20d-78577592fe8a None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:58.814731 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e13b84-b661-4a9a-b055-5a55155c2e25 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:58.834133 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e269c-3693-4564-b20d-78577592fe8a None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:58.911205 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3852a3a-1f43-4a79-a5a0-1202fb627f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:58.912761 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3852a3a-1f43-4a79-a5a0-1202fb627f24 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:58.913253 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3852a3a-1f43-4a79-a5a0-1202fb627f24 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:58.913721 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3852a3a-1f43-4a79-a5a0-1202fb627f24 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:58.916248 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3852a3a-1f43-4a79-a5a0-1202fb627f24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:58.918506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3852a3a-1f43-4a79-a5a0-1202fb627f24 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:58.923039 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 73/142] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:00:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:58.928858 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3852a3a-1f43-4a79-a5a0-1202fb627f24 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:58.971907 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:59.031367 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e13b84-b661-4a9a-b055-5a55155c2e25 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:59.079773 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:59.110658 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa3108-5664-4a1a-8bd2-644a4ca7bf36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:59.111311 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa3108-5664-4a1a-8bd2-644a4ca7bf36 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:59.111621 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa3108-5664-4a1a-8bd2-644a4ca7bf36 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:59.111902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa3108-5664-4a1a-8bd2-644a4ca7bf36 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:59.138390 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaa3108-5664-4a1a-8bd2-644a4ca7bf36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:00:59.145796 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3852a3a-1f43-4a79-a5a0-1202fb627f24 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:59.149031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-afaa3108-5664-4a1a-8bd2-644a4ca7bf36 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 12:00:59.149031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:00:59.149031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:00:59.149031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:00:59.149031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:00:59.149031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:00:59.149031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:00:59.149031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:00:59.149031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:00:59.149031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:00:59.149031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:00:59.149031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:00:59.149031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:00:59.149031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:00:59.149031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:00:59.149031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:00:59.149875 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:00:59.149875 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:00:59.149875 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:00:59.149875 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:00:59.149875 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:00:59.149875 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:00:59.149875 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:00:59.149875 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:00:59.149875 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:00:59.149875 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:00:59.149875 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:00:59.149875 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:00:59.149875 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:00:59.149875 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:00:59.149875 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:00:59.149875 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:00:59.150648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:00:59.150648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:00:59.150648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:00:59.150648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:00:59.150648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:00:59.150648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:00:59.150648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:00:59.150648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:00:59.150648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:00:59.150648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:00:59.150648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 12:00:59.150648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:00:59.156875 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 70/143] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:00:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:00:59.168774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa3108-5664-4a1a-8bd2-644a4ca7bf36 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:59.236970 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5eafdb-fe42-4d33-b0b3-f200dd247323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:59.237532 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5eafdb-fe42-4d33-b0b3-f200dd247323 None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:59.237821 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5eafdb-fe42-4d33-b0b3-f200dd247323 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:59.238104 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5eafdb-fe42-4d33-b0b3-f200dd247323 None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:59.238780 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5eafdb-fe42-4d33-b0b3-f200dd247323 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:59.240383 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5eafdb-fe42-4d33-b0b3-f200dd247323 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:59.330762 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa3108-5664-4a1a-8bd2-644a4ca7bf36 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:59.358293 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 74/144] 10.222.0.98 () {60 vars in 1132 bytes} [Thu Aug 18 12:00:58 2022] POST /identity/v3/users => generated 283 bytes in 430 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:00:59.408293 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd2aa25-528f-4706-b5e7-eee5468901ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:59.409396 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2aa25-528f-4706-b5e7-eee5468901ed None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:59.409879 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2aa25-528f-4706-b5e7-eee5468901ed None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:59.410388 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2aa25-528f-4706-b5e7-eee5468901ed None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:59.411633 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd2aa25-528f-4706-b5e7-eee5468901ed None admin] RBAC: Adding query filter params (name=reader) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:00:59.412237 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd2aa25-528f-4706-b5e7-eee5468901ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:59.414757 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd2aa25-528f-4706-b5e7-eee5468901ed None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:59.417342 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:59.428422 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 71/145] 10.222.0.98 () {56 vars in 1101 bytes} [Thu Aug 18 12:00:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:59.437154 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5eafdb-fe42-4d33-b0b3-f200dd247323 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:59.585378 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5eafdb-fe42-4d33-b0b3-f200dd247323 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:59.616367 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:00:59.628172 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa504b-069f-4ec1-861c-9c4c61747f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:59.628824 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa504b-069f-4ec1-861c-9c4c61747f73 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:59.629114 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa504b-069f-4ec1-861c-9c4c61747f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:59.629398 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa504b-069f-4ec1-861c-9c4c61747f73 None admin] PATH_INFO: `/v3/users/955131e7b5f048a5be55cc222561df18` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:59.630566 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa504b-069f-4ec1-861c-9c4c61747f73 None admin] RBAC: Authorizing `identity:get_user(user_id=955131e7b5f048a5be55cc222561df18)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:59.632064 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa504b-069f-4ec1-861c-9c4c61747f73 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:59.639550 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 75/146] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:00:59 2022] GET /identity/v3/users/955131e7b5f048a5be55cc222561df18 => generated 289 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:59.646116 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd2aa25-528f-4706-b5e7-eee5468901ed None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:59.835906 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd2aa25-528f-4706-b5e7-eee5468901ed None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:00:59.880066 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881574e-2249-4ce5-82fb-91cc24f14a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:00:59.880944 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881574e-2249-4ce5-82fb-91cc24f14a0a None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:59.881403 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881574e-2249-4ce5-82fb-91cc24f14a0a None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:59.881823 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881574e-2249-4ce5-82fb-91cc24f14a0a None admin] PATH_INFO: `/v3/projects/dede24cb1b014e9796b44d6cdac3ed69` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:59.885028 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1881574e-2249-4ce5-82fb-91cc24f14a0a None admin] RBAC: Authorizing `identity:get_project(project_id=dede24cb1b014e9796b44d6cdac3ed69)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:00:59.887219 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1881574e-2249-4ce5-82fb-91cc24f14a0a None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:00:59.889092 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 72/147] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:00:59 2022] GET /identity/v3/projects/dede24cb1b014e9796b44d6cdac3ed69 => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:00:59.898165 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa504b-069f-4ec1-861c-9c4c61747f73 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:00:59.962677 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f3187-7529-42be-93e7-d6f4ead85221 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:59.963205 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f3187-7529-42be-93e7-d6f4ead85221 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:59.963523 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f3187-7529-42be-93e7-d6f4ead85221 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:00:59.965210 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 73/148] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:00:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 12:00:59.981015 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75daeee2-cdba-4599-85ac-f5f2094ea4cc None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:00:59.981613 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75daeee2-cdba-4599-85ac-f5f2094ea4cc None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:00:59.981613 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75daeee2-cdba-4599-85ac-f5f2094ea4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:00.017254 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-75daeee2-cdba-4599-85ac-f5f2094ea4cc None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:00.019292 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-75daeee2-cdba-4599-85ac-f5f2094ea4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:00.050076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa504b-069f-4ec1-861c-9c4c61747f73 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:00.119006 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1293a793-0be8-4b04-8d11-40eb901a41b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:00.120061 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1293a793-0be8-4b04-8d11-40eb901a41b9 None admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:00.120554 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1293a793-0be8-4b04-8d11-40eb901a41b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:00.121615 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1293a793-0be8-4b04-8d11-40eb901a41b9 None admin] PATH_INFO: `/v3/projects/dede24cb1b014e9796b44d6cdac3ed69/users/955131e7b5f048a5be55cc222561df18/roles/93cdab4d25f54b8ca825e7ee5a5dc2d2` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:00.127362 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1293a793-0be8-4b04-8d11-40eb901a41b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=dede24cb1b014e9796b44d6cdac3ed69, user_id=955131e7b5f048a5be55cc222561df18, role_id=93cdab4d25f54b8ca825e7ee5a5dc2d2)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:00.129077 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1293a793-0be8-4b04-8d11-40eb901a41b9 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:00.146698 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 74/149] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:00:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 12:01:00.147811 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:00.208954 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 76/150] 10.222.0.98 () {58 vars in 1439 bytes} [Thu Aug 18 12:00:59 2022] PUT /identity/v3/projects/dede24cb1b014e9796b44d6cdac3ed69/users/955131e7b5f048a5be55cc222561df18/roles/93cdab4d25f54b8ca825e7ee5a5dc2d2 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:01:00.452949 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75daeee2-cdba-4599-85ac-f5f2094ea4cc None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:00.571931 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75daeee2-cdba-4599-85ac-f5f2094ea4cc None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:00.610218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e16e6-f7be-4206-82ef-51bc25bf6f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:00.610867 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e16e6-f7be-4206-82ef-51bc25bf6f4c None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:00.611213 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e16e6-f7be-4206-82ef-51bc25bf6f4c None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:00.611524 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e16e6-f7be-4206-82ef-51bc25bf6f4c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:00.623159 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15e16e6-f7be-4206-82ef-51bc25bf6f4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:00.628042 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-b15e16e6-f7be-4206-82ef-51bc25bf6f4c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:01:00.628042 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:00.628042 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:00.628042 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:00.628042 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:00.628042 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:00.628042 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:00.628042 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:00.628042 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:00.628042 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:00.628042 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:00.628042 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:00.628042 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:00.628042 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:00.628042 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:00.628042 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:00.628902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:00.628902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:00.628902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:00.628902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:00.628902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:00.628902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:00.628902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:00.628902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:00.628902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:00.628902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:00.628902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:00.628902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:00.628902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:00.628902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:00.628902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:00.628902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:00.629656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:00.629656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:00.629656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:00.629656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:00.629656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:00.629656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:00.629656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:00.629656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:00.629656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:00.629656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:00.629656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:01:00.629656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:00.629656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 75/151] 10.222.0.98 () {56 vars in 1096 bytes} [Thu Aug 18 12:01:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:00.635051 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1293a793-0be8-4b04-8d11-40eb901a41b9 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:00.695880 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:00.751319 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1293a793-0be8-4b04-8d11-40eb901a41b9 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:00.792397 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaaba72-3abf-4202-9316-b3da2427e844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:00.793125 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaaba72-3abf-4202-9316-b3da2427e844 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:00.793424 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaaba72-3abf-4202-9316-b3da2427e844 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:00.793711 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaaba72-3abf-4202-9316-b3da2427e844 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:00.794408 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaaba72-3abf-4202-9316-b3da2427e844 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:00.794719 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaaba72-3abf-4202-9316-b3da2427e844 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:00.796151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaaba72-3abf-4202-9316-b3da2427e844 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:00.807352 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 77/152] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:01:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:00.814408 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e16e6-f7be-4206-82ef-51bc25bf6f4c None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:00.902976 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:00.952463 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e16e6-f7be-4206-82ef-51bc25bf6f4c None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:00.994425 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227438cf-1b13-4425-b233-7571b9f7c1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:00.995166 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227438cf-1b13-4425-b233-7571b9f7c1c8 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:00.995344 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227438cf-1b13-4425-b233-7571b9f7c1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:00.995714 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227438cf-1b13-4425-b233-7571b9f7c1c8 None admin] PATH_INFO: `/v3/users/6c92bf2b004241b3b54d40ae3aaaefe8` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:00.996922 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227438cf-1b13-4425-b233-7571b9f7c1c8 None admin] RBAC: Authorizing `identity:get_user(user_id=6c92bf2b004241b3b54d40ae3aaaefe8)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:00.998398 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227438cf-1b13-4425-b233-7571b9f7c1c8 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:01.000017 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 76/153] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:01:00 2022] GET /identity/v3/users/6c92bf2b004241b3b54d40ae3aaaefe8 => generated 247 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:01.006149 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaaba72-3abf-4202-9316-b3da2427e844 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:01.120227 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaaba72-3abf-4202-9316-b3da2427e844 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:01.135798 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:01.157696 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e920affe-4b20-42d2-8def-c50a5e6f1cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:01.158332 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920affe-4b20-42d2-8def-c50a5e6f1cae None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:01.158621 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920affe-4b20-42d2-8def-c50a5e6f1cae None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:01.158901 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920affe-4b20-42d2-8def-c50a5e6f1cae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:01.162381 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82125cd-526d-4c98-ae02-7345761c8add None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:01.162727 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82125cd-526d-4c98-ae02-7345761c8add None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:01.163006 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82125cd-526d-4c98-ae02-7345761c8add None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:01.164390 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 77/154] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 12:01:01.169872 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb8ed4-5787-4fdd-84f1-f27b2ba3b31d None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:01.170208 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb8ed4-5787-4fdd-84f1-f27b2ba3b31d None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:01.170490 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb8ed4-5787-4fdd-84f1-f27b2ba3b31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:01.196808 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-1ddb8ed4-5787-4fdd-84f1-f27b2ba3b31d None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:01.198099 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-1ddb8ed4-5787-4fdd-84f1-f27b2ba3b31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:01.313050 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e920affe-4b20-42d2-8def-c50a5e6f1cae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:01.315881 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e920affe-4b20-42d2-8def-c50a5e6f1cae None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:01.385668 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 78/155] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 12:01:01.387187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae6020-9aae-48a0-8160-9e6cc2758267 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:01.387647 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae6020-9aae-48a0-8160-9e6cc2758267 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:01.388069 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae6020-9aae-48a0-8160-9e6cc2758267 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:01.389506 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 79/156] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:01.400262 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52fd5c-e436-4135-9df3-e3206eaf262b None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:01.400851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52fd5c-e436-4135-9df3-e3206eaf262b None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:01.401276 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52fd5c-e436-4135-9df3-e3206eaf262b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:01.436458 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-0c52fd5c-e436-4135-9df3-e3206eaf262b None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:01.437911 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-0c52fd5c-e436-4135-9df3-e3206eaf262b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:01.501804 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:01.507740 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 78/157] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:01:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:01:01.514035 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e920affe-4b20-42d2-8def-c50a5e6f1cae None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:01.523108 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 80/158] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 12:01:01.547799 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:01.658002 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e920affe-4b20-42d2-8def-c50a5e6f1cae None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:01.704992 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba9fe47-5cda-40c0-8ee6-00395b066dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:01.705648 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9fe47-5cda-40c0-8ee6-00395b066dfd None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:01.705940 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9fe47-5cda-40c0-8ee6-00395b066dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:01.706217 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9fe47-5cda-40c0-8ee6-00395b066dfd None admin] PATH_INFO: `/v3/projects/dede24cb1b014e9796b44d6cdac3ed69` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:01.707449 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba9fe47-5cda-40c0-8ee6-00395b066dfd None admin] RBAC: Authorizing `identity:get_project(project_id=dede24cb1b014e9796b44d6cdac3ed69)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:01.708932 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba9fe47-5cda-40c0-8ee6-00395b066dfd None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:01.711020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 79/159] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:01:01 2022] GET /identity/v3/projects/dede24cb1b014e9796b44d6cdac3ed69 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:01.717183 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c52fd5c-e436-4135-9df3-e3206eaf262b None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:01.860329 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c52fd5c-e436-4135-9df3-e3206eaf262b None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:01.898105 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec97c6-63c2-4ed5-b872-995a1d92fcda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:01.899102 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec97c6-63c2-4ed5-b872-995a1d92fcda None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:01.899630 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec97c6-63c2-4ed5-b872-995a1d92fcda None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:01.900107 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec97c6-63c2-4ed5-b872-995a1d92fcda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:01.901033 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec97c6-63c2-4ed5-b872-995a1d92fcda None admin] RBAC: Adding query filter params (role.id=c53a0e13b58642f291c856efe1f0b87d, scope.project.id=dede24cb1b014e9796b44d6cdac3ed69, user.id=6c92bf2b004241b3b54d40ae3aaaefe8) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:01.901544 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec97c6-63c2-4ed5-b872-995a1d92fcda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:01.903248 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec97c6-63c2-4ed5-b872-995a1d92fcda None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:01.909962 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 81/160] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:01:01 2022] GET /identity/v3/role_assignments?user.id=6c92bf2b004241b3b54d40ae3aaaefe8&scope.project.id=dede24cb1b014e9796b44d6cdac3ed69&role.id=c53a0e13b58642f291c856efe1f0b87d => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:02.073605 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e33a86-ea4c-46bf-a6f1-91be99ded86a None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:02.074886 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e33a86-ea4c-46bf-a6f1-91be99ded86a None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:02.075371 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e33a86-ea4c-46bf-a6f1-91be99ded86a None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:02.077052 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 80/161] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:02.082955 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55fe2f2-f07b-433f-bb8c-485b1486904c None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:02.083456 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55fe2f2-f07b-433f-bb8c-485b1486904c None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:02.083533 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55fe2f2-f07b-433f-bb8c-485b1486904c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:02.089014 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e33a86-ea4c-46bf-a6f1-91be99ded86a None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:02.108582 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-a55fe2f2-f07b-433f-bb8c-485b1486904c None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:02.109806 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-a55fe2f2-f07b-433f-bb8c-485b1486904c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:02.184435 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:02.216982 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 82/162] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:02.217866 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55fe2f2-f07b-433f-bb8c-485b1486904c None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:02.288835 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:02.331957 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55fe2f2-f07b-433f-bb8c-485b1486904c None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:02.340641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e33a86-ea4c-46bf-a6f1-91be99ded86a None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:02.372508 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2c71f-a00d-414e-84b7-0c51c78ac0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:02.373155 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2c71f-a00d-414e-84b7-0c51c78ac0e0 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:02.373440 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2c71f-a00d-414e-84b7-0c51c78ac0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:02.373717 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2c71f-a00d-414e-84b7-0c51c78ac0e0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:02.396189 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c1565-b3a0-4839-acb6-82ea9593c068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:02.397073 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c1565-b3a0-4839-acb6-82ea9593c068 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:02.397499 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c1565-b3a0-4839-acb6-82ea9593c068 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:02.398073 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c1565-b3a0-4839-acb6-82ea9593c068 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:02.400779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c2c71f-a00d-414e-84b7-0c51c78ac0e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:02.405666 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-f0c2c71f-a00d-414e-84b7-0c51c78ac0e0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 12:01:02.405666 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:02.405666 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:02.405666 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:02.405666 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:02.405666 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:02.405666 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:02.405666 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:02.405666 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:02.405666 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:02.405666 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:02.405666 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:02.405666 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:02.405666 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:02.405666 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:02.405666 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:02.406815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:02.406815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:02.406815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:02.406815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:02.406815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:02.406815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:02.406815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:02.406815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:02.406815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:02.406815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:02.406815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:02.406815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:02.406815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:02.406815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:02.406815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:02.406815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:02.407864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:02.407864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:02.407864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:02.407864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:02.407864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:02.407864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:02.407864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:02.407864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:02.407864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:02.407864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:02.407864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 12:01:02.407864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:02.407864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 83/163] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:01:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:02.412880 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2c71f-a00d-414e-84b7-0c51c78ac0e0 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:02.413011 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086c1565-b3a0-4839-acb6-82ea9593c068 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:02.418800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-086c1565-b3a0-4839-acb6-82ea9593c068 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 12:01:02.418800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:02.418800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:02.418800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:02.418800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:02.418800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:02.418800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:02.418800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:02.418800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:02.418800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:02.418800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:02.418800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:02.418800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:02.418800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:02.418800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:02.418800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:02.419679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:02.419679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:02.419679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:02.419679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:02.419679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:02.419679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:02.419679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:02.419679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:02.419679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:02.419679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:02.419679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:02.419679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:02.419679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:02.419679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:02.419679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:02.419679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:02.420462 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:02.420462 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:02.420462 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:02.420462 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:02.420462 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:02.420462 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:02.420462 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:02.420462 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:02.420462 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:02.420462 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:02.420462 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 12:01:02.420462 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:02.422409 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 81/164] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:01:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:02.431616 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c1565-b3a0-4839-acb6-82ea9593c068 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:02.504124 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:02.571838 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c1565-b3a0-4839-acb6-82ea9593c068 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:02.629511 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12911542-d72f-4600-ae11-7e12034b1add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:02.630139 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12911542-d72f-4600-ae11-7e12034b1add None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:02.630423 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12911542-d72f-4600-ae11-7e12034b1add None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:02.630701 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12911542-d72f-4600-ae11-7e12034b1add None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:02.631380 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12911542-d72f-4600-ae11-7e12034b1add None admin] RBAC: Adding query filter params (name=member) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:02.631866 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12911542-d72f-4600-ae11-7e12034b1add None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:02.633270 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12911542-d72f-4600-ae11-7e12034b1add None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:02.633543 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2c71f-a00d-414e-84b7-0c51c78ac0e0 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:02.650152 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 82/165] 10.222.0.98 () {56 vars in 1101 bytes} [Thu Aug 18 12:01:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:01:02.658813 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12911542-d72f-4600-ae11-7e12034b1add None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:02.664033 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:02.712304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cadbd0-c7db-444f-8da9-587f3441fdbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:02.713093 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cadbd0-c7db-444f-8da9-587f3441fdbe None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:02.713374 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cadbd0-c7db-444f-8da9-587f3441fdbe None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:02.713694 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cadbd0-c7db-444f-8da9-587f3441fdbe None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:02.716705 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cadbd0-c7db-444f-8da9-587f3441fdbe None admin] RBAC: Adding query filter params (name=member) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:02.716705 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cadbd0-c7db-444f-8da9-587f3441fdbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:02.716907 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cadbd0-c7db-444f-8da9-587f3441fdbe None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:02.728991 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 84/166] 10.222.0.98 () {56 vars in 1101 bytes} [Thu Aug 18 12:01:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:02.735220 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cadbd0-c7db-444f-8da9-587f3441fdbe None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:02.797098 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12911542-d72f-4600-ae11-7e12034b1add None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:02.879866 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a881d40-227c-4a5d-8862-12982654f3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:02.880604 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a881d40-227c-4a5d-8862-12982654f3c5 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:02.880959 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a881d40-227c-4a5d-8862-12982654f3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:02.881328 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a881d40-227c-4a5d-8862-12982654f3c5 None admin] PATH_INFO: `/v3/users/31feea8cb0e3454194acf739392519e2` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:02.888129 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a881d40-227c-4a5d-8862-12982654f3c5 None admin] RBAC: Authorizing `identity:get_user(user_id=31feea8cb0e3454194acf739392519e2)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:02.889860 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a881d40-227c-4a5d-8862-12982654f3c5 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:02.891585 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 83/167] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:01:02 2022] GET /identity/v3/users/31feea8cb0e3454194acf739392519e2 => generated 275 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:01:02.897633 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a881d40-227c-4a5d-8862-12982654f3c5 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:02.905185 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:02.913781 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cadbd0-c7db-444f-8da9-587f3441fdbe None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:02.970867 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ad2b7e-624d-49dc-91ce-503176c70340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:02.971552 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad2b7e-624d-49dc-91ce-503176c70340 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:02.971840 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad2b7e-624d-49dc-91ce-503176c70340 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:02.972121 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad2b7e-624d-49dc-91ce-503176c70340 None admin] PATH_INFO: `/v3/users/31feea8cb0e3454194acf739392519e2` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:02.973380 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ad2b7e-624d-49dc-91ce-503176c70340 None admin] RBAC: Authorizing `identity:get_user(user_id=31feea8cb0e3454194acf739392519e2)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:02.974915 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ad2b7e-624d-49dc-91ce-503176c70340 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:02.976753 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 85/168] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:01:02 2022] GET /identity/v3/users/31feea8cb0e3454194acf739392519e2 => generated 275 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:02.977955 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ad2b7e-624d-49dc-91ce-503176c70340 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:03.052023 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:03.080960 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ad2b7e-624d-49dc-91ce-503176c70340 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:03.090896 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a881d40-227c-4a5d-8862-12982654f3c5 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:03.138398 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3a7b9f-45d2-438c-9b64-64700246cbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:03.139106 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3a7b9f-45d2-438c-9b64-64700246cbb7 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:03.139302 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3a7b9f-45d2-438c-9b64-64700246cbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:03.139628 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3a7b9f-45d2-438c-9b64-64700246cbb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:03.162936 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd90f9-dc26-4575-ad7d-7caa63981bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:03.163621 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd90f9-dc26-4575-ad7d-7caa63981bfa None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:03.163910 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd90f9-dc26-4575-ad7d-7caa63981bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:03.164188 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd90f9-dc26-4575-ad7d-7caa63981bfa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:03.175888 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:03.188426 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfd90f9-dc26-4575-ad7d-7caa63981bfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:03.193900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-bcfd90f9-dc26-4575-ad7d-7caa63981bfa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 12:01:03.193900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:03.193900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:03.193900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:03.193900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:03.193900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:03.193900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:03.193900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:03.193900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:03.193900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:03.193900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:03.193900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:03.193900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:03.193900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:03.193900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:03.193900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:03.194893 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:03.194893 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:03.194893 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:03.194893 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:03.194893 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:03.194893 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:03.194893 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:03.194893 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:03.194893 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:03.194893 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:03.194893 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:03.194893 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:03.194893 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:03.194893 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:03.194893 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:03.194893 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:03.195737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:03.195737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:03.195737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:03.195737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:03.195737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:03.195737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:03.195737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:03.195737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:03.195737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:03.195737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:03.195737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 12:01:03.195737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:03.195737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 84/169] 10.222.0.98 () {56 vars in 1114 bytes} [Thu Aug 18 12:01:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:03.202493 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd90f9-dc26-4575-ad7d-7caa63981bfa None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:03.214869 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3a7b9f-45d2-438c-9b64-64700246cbb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:03.216997 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3a7b9f-45d2-438c-9b64-64700246cbb7 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:03.419918 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 86/170] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:01:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:01:03.419918 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3a7b9f-45d2-438c-9b64-64700246cbb7 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:03.471328 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd90f9-dc26-4575-ad7d-7caa63981bfa None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:03.502069 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:03.506864 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e67cc3-47e0-41ca-a69c-11800760150a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:03.507525 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e67cc3-47e0-41ca-a69c-11800760150a None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:03.507815 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e67cc3-47e0-41ca-a69c-11800760150a None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:03.508098 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e67cc3-47e0-41ca-a69c-11800760150a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:03.537983 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:03.578569 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e67cc3-47e0-41ca-a69c-11800760150a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:03.580219 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e67cc3-47e0-41ca-a69c-11800760150a None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:03.625357 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3a7b9f-45d2-438c-9b64-64700246cbb7 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:03.687630 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 85/171] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:01:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:01:03.688489 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e67cc3-47e0-41ca-a69c-11800760150a None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:03.721008 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:03.726764 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0695e-f5fd-4b00-970e-4c03300344a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:03.728173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0695e-f5fd-4b00-970e-4c03300344a8 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:03.728840 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0695e-f5fd-4b00-970e-4c03300344a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:03.735605 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0695e-f5fd-4b00-970e-4c03300344a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:03.735605 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0695e-f5fd-4b00-970e-4c03300344a8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:03.735605 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0695e-f5fd-4b00-970e-4c03300344a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:03.735605 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0695e-f5fd-4b00-970e-4c03300344a8 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:03.755757 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 87/172] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:01:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:03.756652 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0695e-f5fd-4b00-970e-4c03300344a8 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:03.798335 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e67cc3-47e0-41ca-a69c-11800760150a None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:03.834512 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675ee18-db36-4e5c-8b08-b50872320f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:03.835167 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675ee18-db36-4e5c-8b08-b50872320f9b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:03.835495 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675ee18-db36-4e5c-8b08-b50872320f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:03.835780 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675ee18-db36-4e5c-8b08-b50872320f9b None admin] PATH_INFO: `/v3/projects/dede24cb1b014e9796b44d6cdac3ed69` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:03.836952 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3675ee18-db36-4e5c-8b08-b50872320f9b None admin] RBAC: Authorizing `identity:get_project(project_id=dede24cb1b014e9796b44d6cdac3ed69)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:03.838412 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3675ee18-db36-4e5c-8b08-b50872320f9b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:03.840508 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 86/173] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:01:03 2022] GET /identity/v3/projects/dede24cb1b014e9796b44d6cdac3ed69 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:03.841307 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675ee18-db36-4e5c-8b08-b50872320f9b None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:03.967741 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675ee18-db36-4e5c-8b08-b50872320f9b None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:03.969924 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:03.979559 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0695e-f5fd-4b00-970e-4c03300344a8 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:04.020773 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01293525-fdb0-4ccb-a01e-393dd7397af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:04.021445 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01293525-fdb0-4ccb-a01e-393dd7397af8 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:04.022354 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01293525-fdb0-4ccb-a01e-393dd7397af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:04.022642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01293525-fdb0-4ccb-a01e-393dd7397af8 None admin] PATH_INFO: `/v3/users/31feea8cb0e3454194acf739392519e2` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:04.023889 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01293525-fdb0-4ccb-a01e-393dd7397af8 None admin] RBAC: Authorizing `identity:get_user(user_id=31feea8cb0e3454194acf739392519e2)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:04.025940 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01293525-fdb0-4ccb-a01e-393dd7397af8 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:04.029722 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 87/174] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:01:03 2022] GET /identity/v3/users/31feea8cb0e3454194acf739392519e2 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:04.031131 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:04.032576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927e885-b0b7-4246-8632-c087152b74e2 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:04.032868 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927e885-b0b7-4246-8632-c087152b74e2 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:04.033145 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927e885-b0b7-4246-8632-c087152b74e2 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:04.034563 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 88/175] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:04.035391 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0927e885-b0b7-4246-8632-c087152b74e2 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:04.051050 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5938200b-7370-4a26-99d8-ac2f2a3faaf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:04.051744 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938200b-7370-4a26-99d8-ac2f2a3faaf3 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:04.052028 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938200b-7370-4a26-99d8-ac2f2a3faaf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:04.052302 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938200b-7370-4a26-99d8-ac2f2a3faaf3 None admin] PATH_INFO: `/v3/projects/10871d2fec9e407298cba25d8c3c6a3c` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:04.053615 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5938200b-7370-4a26-99d8-ac2f2a3faaf3 None admin] RBAC: Authorizing `identity:get_project(project_id=10871d2fec9e407298cba25d8c3c6a3c)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:04.055139 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5938200b-7370-4a26-99d8-ac2f2a3faaf3 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:04.057059 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 88/176] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:01:03 2022] GET /identity/v3/projects/10871d2fec9e407298cba25d8c3c6a3c => generated 312 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:04.065321 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ae4ca-23bf-4fc2-b06d-13e8f41f3139 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:04.065943 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ae4ca-23bf-4fc2-b06d-13e8f41f3139 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:04.066423 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ae4ca-23bf-4fc2-b06d-13e8f41f3139 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:04.068204 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 89/177] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:04 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:04.069889 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d3fed-4f03-475f-aa3b-fd5db9bc1da3 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:04.070178 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d3fed-4f03-475f-aa3b-fd5db9bc1da3 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:04.070455 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d3fed-4f03-475f-aa3b-fd5db9bc1da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:04.091299 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-233d3fed-4f03-475f-aa3b-fd5db9bc1da3 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:04.092661 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-233d3fed-4f03-475f-aa3b-fd5db9bc1da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:04.165703 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0927e885-b0b7-4246-8632-c087152b74e2 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:04.203751 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 90/178] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:04.204831 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d3fed-4f03-475f-aa3b-fd5db9bc1da3 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:04.222405 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce62817-4d37-46ec-97eb-8c95bb5be5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:04.222955 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce62817-4d37-46ec-97eb-8c95bb5be5eb None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:04.223243 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce62817-4d37-46ec-97eb-8c95bb5be5eb None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:04.223548 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce62817-4d37-46ec-97eb-8c95bb5be5eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:04.224292 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce62817-4d37-46ec-97eb-8c95bb5be5eb None admin] RBAC: Adding query filter params (role.id=c81e28908b9b49a38d33978df4d11716, scope.project.id=dede24cb1b014e9796b44d6cdac3ed69, user.id=31feea8cb0e3454194acf739392519e2) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:04.224607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce62817-4d37-46ec-97eb-8c95bb5be5eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:04.226080 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce62817-4d37-46ec-97eb-8c95bb5be5eb None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:04.239083 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 89/179] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:01:04 2022] GET /identity/v3/role_assignments?user.id=31feea8cb0e3454194acf739392519e2&scope.project.id=dede24cb1b014e9796b44d6cdac3ed69&role.id=c81e28908b9b49a38d33978df4d11716 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:04.240049 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce62817-4d37-46ec-97eb-8c95bb5be5eb None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:04.307156 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:04.376288 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d3fed-4f03-475f-aa3b-fd5db9bc1da3 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:04.422624 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:04.426571 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce62817-4d37-46ec-97eb-8c95bb5be5eb None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:04.433094 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c88de05-f651-40e5-8ddd-022251ae4fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:04.437140 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88de05-f651-40e5-8ddd-022251ae4fc6 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:04.437140 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88de05-f651-40e5-8ddd-022251ae4fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:04.437140 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88de05-f651-40e5-8ddd-022251ae4fc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:04.500260 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b36b6a-bed7-4ab2-b5d2-102a9d81c0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:04.500927 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b36b6a-bed7-4ab2-b5d2-102a9d81c0e9 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:04.501091 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b36b6a-bed7-4ab2-b5d2-102a9d81c0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:04.501353 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b36b6a-bed7-4ab2-b5d2-102a9d81c0e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:04.502106 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b36b6a-bed7-4ab2-b5d2-102a9d81c0e9 None admin] RBAC: Adding query filter params (role.id=c81e28908b9b49a38d33978df4d11716, scope.project.id=10871d2fec9e407298cba25d8c3c6a3c, user.id=31feea8cb0e3454194acf739392519e2) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:04.502422 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b36b6a-bed7-4ab2-b5d2-102a9d81c0e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:04.503938 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b36b6a-bed7-4ab2-b5d2-102a9d81c0e9 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:04.515038 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 90/180] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:01:04 2022] GET /identity/v3/role_assignments?user.id=31feea8cb0e3454194acf739392519e2&scope.project.id=10871d2fec9e407298cba25d8c3c6a3c&role.id=c81e28908b9b49a38d33978df4d11716 => generated 608 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:04.520052 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a412e-afb1-4cd5-b135-9474579916ad None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:04.520345 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a412e-afb1-4cd5-b135-9474579916ad None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:04.520611 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a412e-afb1-4cd5-b135-9474579916ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:04.552606 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-ce5a412e-afb1-4cd5-b135-9474579916ad None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:04.553879 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-ce5a412e-afb1-4cd5-b135-9474579916ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:04.565298 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c88de05-f651-40e5-8ddd-022251ae4fc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:04.566823 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c88de05-f651-40e5-8ddd-022251ae4fc6 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:04.704204 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 91/181] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:04.705872 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92573e-79ee-4c3d-8ecd-21dfffa445fe None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:04.706368 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92573e-79ee-4c3d-8ecd-21dfffa445fe None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:04.706871 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92573e-79ee-4c3d-8ecd-21dfffa445fe None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:04.708805 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 92/182] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:04.713972 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b92573e-79ee-4c3d-8ecd-21dfffa445fe None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:04.734523 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 91/183] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:01:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:01:04.739740 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92394413-f621-439f-9c40-cfe798f32143 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:04.740407 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92394413-f621-439f-9c40-cfe798f32143 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:04.740701 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92394413-f621-439f-9c40-cfe798f32143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:04.749176 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:04.776522 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-92394413-f621-439f-9c40-cfe798f32143 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:04.778246 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-92394413-f621-439f-9c40-cfe798f32143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:04.811325 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:04.842561 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b92573e-79ee-4c3d-8ecd-21dfffa445fe None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:04.870490 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 92/184] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:04.873973 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92394413-f621-439f-9c40-cfe798f32143 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:04.889765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f326a-834d-4709-9ea9-cf663a7d1900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:04.890531 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f326a-834d-4709-9ea9-cf663a7d1900 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:04.890821 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f326a-834d-4709-9ea9-cf663a7d1900 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:04.891098 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f326a-834d-4709-9ea9-cf663a7d1900 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:04.903575 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80f326a-834d-4709-9ea9-cf663a7d1900 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:04.909199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-d80f326a-834d-4709-9ea9-cf663a7d1900 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:01:04.909199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:04.909199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:04.909199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:04.909199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:04.909199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:04.909199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:04.909199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:04.909199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:04.909199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:04.909199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:04.909199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:04.909199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:04.909199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:04.909199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:04.909199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:04.910059 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:04.910059 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:04.910059 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:04.910059 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:04.910059 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:04.910059 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:04.910059 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:04.910059 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:04.910059 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:04.910059 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:04.910059 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:04.910059 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:04.910059 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:04.910059 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:04.910059 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:04.910059 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:04.910840 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:04.910840 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:04.910840 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:04.910840 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:04.910840 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:04.910840 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:04.910840 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:04.910840 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:04.910840 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:04.910840 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:04.910840 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:01:04.910840 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:04.910840 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 93/185] 10.222.0.98 () {56 vars in 1096 bytes} [Thu Aug 18 12:01:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:04.914266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fffd95-93fc-4e9b-adc5-2aabd61f8d08 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:04.914266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fffd95-93fc-4e9b-adc5-2aabd61f8d08 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:04.914266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fffd95-93fc-4e9b-adc5-2aabd61f8d08 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:04.914266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 94/186] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:04.918793 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fffd95-93fc-4e9b-adc5-2aabd61f8d08 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:04.981700 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92394413-f621-439f-9c40-cfe798f32143 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:05.016510 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:05.018027 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5462ed-3a42-49df-8e21-1979f6d86a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:05.018742 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5462ed-3a42-49df-8e21-1979f6d86a59 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:05.019119 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5462ed-3a42-49df-8e21-1979f6d86a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:05.019735 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5462ed-3a42-49df-8e21-1979f6d86a59 None admin] PATH_INFO: `/v3/projects/dede24cb1b014e9796b44d6cdac3ed69` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:05.021065 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5462ed-3a42-49df-8e21-1979f6d86a59 None admin] RBAC: Authorizing `identity:get_project(project_id=dede24cb1b014e9796b44d6cdac3ed69)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:05.022788 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5462ed-3a42-49df-8e21-1979f6d86a59 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:05.024639 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 93/187] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:01:04 2022] GET /identity/v3/projects/dede24cb1b014e9796b44d6cdac3ed69 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:05.025813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f10c99-4983-4140-bb13-d9ad45efbd9c None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:05.026100 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f10c99-4983-4140-bb13-d9ad45efbd9c None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:05.026380 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f10c99-4983-4140-bb13-d9ad45efbd9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:05.034135 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fffd95-93fc-4e9b-adc5-2aabd61f8d08 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:05.046828 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-54f10c99-4983-4140-bb13-d9ad45efbd9c None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:05.048079 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-54f10c99-4983-4140-bb13-d9ad45efbd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:05.067936 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:05.072508 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff980c-2b8b-47ef-90a4-dfd8cc8cb10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:05.073419 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff980c-2b8b-47ef-90a4-dfd8cc8cb10f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:05.073685 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff980c-2b8b-47ef-90a4-dfd8cc8cb10f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:05.074071 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff980c-2b8b-47ef-90a4-dfd8cc8cb10f None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:05.075354 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff980c-2b8b-47ef-90a4-dfd8cc8cb10f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:05.075708 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff980c-2b8b-47ef-90a4-dfd8cc8cb10f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:05.080169 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff980c-2b8b-47ef-90a4-dfd8cc8cb10f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:05.092243 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 95/188] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:01:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:05.093156 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff980c-2b8b-47ef-90a4-dfd8cc8cb10f None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:05.125820 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 94/189] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:05.126683 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f10c99-4983-4140-bb13-d9ad45efbd9c None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:05.209055 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff980c-2b8b-47ef-90a4-dfd8cc8cb10f None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:05.243112 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f10c99-4983-4140-bb13-d9ad45efbd9c None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:05.244934 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960d3872-0e09-43b7-a1c3-aa1e3f7c0dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:05.245581 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d3872-0e09-43b7-a1c3-aa1e3f7c0dac None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:05.245868 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d3872-0e09-43b7-a1c3-aa1e3f7c0dac None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:05.246152 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d3872-0e09-43b7-a1c3-aa1e3f7c0dac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:05.259719 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960d3872-0e09-43b7-a1c3-aa1e3f7c0dac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:05.265732 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-960d3872-0e09-43b7-a1c3-aa1e3f7c0dac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 12:01:05.265732 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:05.265732 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:05.265732 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:05.265732 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:05.265732 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:05.265732 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:05.265732 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:05.265732 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:05.265732 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:05.265732 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:05.265732 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:05.265732 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:05.265732 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:05.265732 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:05.265732 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:05.266641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:05.266641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:05.266641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:05.266641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:05.266641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:05.266641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:05.266641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:05.266641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:05.266641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:05.266641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:05.266641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:05.266641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:05.266641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:05.266641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:05.266641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:05.266641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:05.267453 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:05.267453 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:05.267453 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:05.267453 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:05.267453 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:05.267453 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:05.267453 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:05.267453 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:05.267453 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:05.267453 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:05.267453 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 12:01:05.267453 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:05.267453 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 96/190] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:01:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:05.268132 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960d3872-0e09-43b7-a1c3-aa1e3f7c0dac None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:05.279749 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:05.293581 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4f1c2-1a62-4858-a193-97e0dd245114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:05.293581 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4f1c2-1a62-4858-a193-97e0dd245114 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:05.293581 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4f1c2-1a62-4858-a193-97e0dd245114 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:05.293581 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4f1c2-1a62-4858-a193-97e0dd245114 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:05.293999 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a4f1c2-1a62-4858-a193-97e0dd245114 None admin] RBAC: Adding query filter params (role.id=3a4b7e53b8644e508ac440f71138be16, scope.project.id=dede24cb1b014e9796b44d6cdac3ed69, user.id=31feea8cb0e3454194acf739392519e2) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:05.293999 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a4f1c2-1a62-4858-a193-97e0dd245114 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:05.293999 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a4f1c2-1a62-4858-a193-97e0dd245114 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:05.297843 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 95/191] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:01:05 2022] GET /identity/v3/role_assignments?user.id=31feea8cb0e3454194acf739392519e2&scope.project.id=dede24cb1b014e9796b44d6cdac3ed69&role.id=3a4b7e53b8644e508ac440f71138be16 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:05.298641 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4f1c2-1a62-4858-a193-97e0dd245114 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:05.324991 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:05.410221 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960d3872-0e09-43b7-a1c3-aa1e3f7c0dac None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:05.446430 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4f1c2-1a62-4858-a193-97e0dd245114 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:05.457521 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9298959-9fe0-4f6c-942d-6b5b4eec3a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:05.458706 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9298959-9fe0-4f6c-942d-6b5b4eec3a07 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:05.458706 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9298959-9fe0-4f6c-942d-6b5b4eec3a07 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:05.458706 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9298959-9fe0-4f6c-942d-6b5b4eec3a07 None admin] PATH_INFO: `/v3/users/15d11995738c4ded976589903d34b89e` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:05.481223 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9298959-9fe0-4f6c-942d-6b5b4eec3a07 None admin] RBAC: Authorizing `identity:get_user(user_id=15d11995738c4ded976589903d34b89e)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:05.481629 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9298959-9fe0-4f6c-942d-6b5b4eec3a07 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:05.483321 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 97/192] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:01:05 2022] GET /identity/v3/users/15d11995738c4ded976589903d34b89e => generated 283 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:05.488310 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9298959-9fe0-4f6c-942d-6b5b4eec3a07 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:05.491534 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c680b-c488-439d-8443-e03ac9611e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:05.491773 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c680b-c488-439d-8443-e03ac9611e1a None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:05.491998 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c680b-c488-439d-8443-e03ac9611e1a None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:05.492279 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c680b-c488-439d-8443-e03ac9611e1a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:05.506469 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43c680b-c488-439d-8443-e03ac9611e1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:05.511813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-c43c680b-c488-439d-8443-e03ac9611e1a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 12:01:05.511813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:05.511813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:05.511813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:05.511813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:05.511813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:05.511813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:05.511813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:05.511813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:05.511813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:05.511813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:05.511813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:05.511813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:05.511813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:05.511813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:05.511813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:05.513425 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:05.513425 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:05.513425 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:05.513425 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:05.513425 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:05.513425 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:05.513425 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:05.513425 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:05.513425 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:05.513425 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:05.513425 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:05.513425 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:05.513425 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:05.513425 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:05.513425 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:05.513425 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:05.514423 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:05.514423 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:05.514423 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:05.514423 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:05.514423 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:05.514423 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:05.514423 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:05.514423 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:05.514423 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:05.514423 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:05.514423 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 12:01:05.514423 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:05.515367 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 96/193] 10.222.0.98 () {56 vars in 1114 bytes} [Thu Aug 18 12:01:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:05.516153 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c680b-c488-439d-8443-e03ac9611e1a None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:05.572557 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:05.600170 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:05.607819 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9298959-9fe0-4f6c-942d-6b5b4eec3a07 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:05.620937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c680b-c488-439d-8443-e03ac9611e1a None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:05.643496 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244db3cc-ad55-41ce-a805-70b6ab40d989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:05.644115 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244db3cc-ad55-41ce-a805-70b6ab40d989 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:05.644654 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244db3cc-ad55-41ce-a805-70b6ab40d989 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:05.644921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244db3cc-ad55-41ce-a805-70b6ab40d989 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:05.645588 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244db3cc-ad55-41ce-a805-70b6ab40d989 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:05.645892 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244db3cc-ad55-41ce-a805-70b6ab40d989 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:05.647265 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244db3cc-ad55-41ce-a805-70b6ab40d989 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:05.655406 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 98/194] 10.222.0.98 () {56 vars in 1101 bytes} [Thu Aug 18 12:01:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:05.656456 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb71aac-4990-4392-bcda-26107777bc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:05.657067 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb71aac-4990-4392-bcda-26107777bc91 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:05.657350 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb71aac-4990-4392-bcda-26107777bc91 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:05.657683 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb71aac-4990-4392-bcda-26107777bc91 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:05.658815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb71aac-4990-4392-bcda-26107777bc91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:05.660314 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb71aac-4990-4392-bcda-26107777bc91 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:05.660737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244db3cc-ad55-41ce-a805-70b6ab40d989 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:05.663542 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 97/195] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:01:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:05.663542 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb71aac-4990-4392-bcda-26107777bc91 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:05.761314 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb71aac-4990-4392-bcda-26107777bc91 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:05.766377 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244db3cc-ad55-41ce-a805-70b6ab40d989 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:05.792102 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:05.794489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a4842-9c57-487c-acee-8b0f79b2929d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:05.795106 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a4842-9c57-487c-acee-8b0f79b2929d None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:05.795386 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a4842-9c57-487c-acee-8b0f79b2929d None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:05.795696 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a4842-9c57-487c-acee-8b0f79b2929d None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:05.796416 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a4842-9c57-487c-acee-8b0f79b2929d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:05.796720 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a4842-9c57-487c-acee-8b0f79b2929d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:05.798091 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a4842-9c57-487c-acee-8b0f79b2929d None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:05.799827 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c063dab3-74b8-4f07-8a82-ac7841dd45fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:05.800782 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063dab3-74b8-4f07-8a82-ac7841dd45fd None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:05.801071 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063dab3-74b8-4f07-8a82-ac7841dd45fd None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:05.801351 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063dab3-74b8-4f07-8a82-ac7841dd45fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:05.807110 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 98/196] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:01:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:05.807806 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a4842-9c57-487c-acee-8b0f79b2929d None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:05.823105 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:05.872708 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c063dab3-74b8-4f07-8a82-ac7841dd45fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:05.874176 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c063dab3-74b8-4f07-8a82-ac7841dd45fd None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:05.912768 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a4842-9c57-487c-acee-8b0f79b2929d None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:05.946977 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c152923b-c6f4-47ac-9295-bade1455ec6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:05.947642 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152923b-c6f4-47ac-9295-bade1455ec6f None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:05.947924 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152923b-c6f4-47ac-9295-bade1455ec6f None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:05.948199 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152923b-c6f4-47ac-9295-bade1455ec6f None admin] PATH_INFO: `/v3/users/955131e7b5f048a5be55cc222561df18` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:05.949335 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c152923b-c6f4-47ac-9295-bade1455ec6f None admin] RBAC: Authorizing `identity:get_user(user_id=955131e7b5f048a5be55cc222561df18)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:05.950809 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c152923b-c6f4-47ac-9295-bade1455ec6f None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:05.952428 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 99/197] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:01:05 2022] GET /identity/v3/users/955131e7b5f048a5be55cc222561df18 => generated 289 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:05.952428 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c152923b-c6f4-47ac-9295-bade1455ec6f None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:05.979636 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 99/198] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:01:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:01:05.980338 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c063dab3-74b8-4f07-8a82-ac7841dd45fd None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:06.008461 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:06.034906 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:06.057684 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c152923b-c6f4-47ac-9295-bade1455ec6f None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:06.080784 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c063dab3-74b8-4f07-8a82-ac7841dd45fd None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:06.091257 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d8815-3bc0-42f9-b800-b767cfd28bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:06.091971 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d8815-3bc0-42f9-b800-b767cfd28bb0 None admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:06.092258 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d8815-3bc0-42f9-b800-b767cfd28bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:06.092535 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d8815-3bc0-42f9-b800-b767cfd28bb0 None admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:06.093177 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02d8815-3bc0-42f9-b800-b767cfd28bb0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:06.094622 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02d8815-3bc0-42f9-b800-b767cfd28bb0 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:06.114763 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd8779-1385-446f-b6bb-f38ea8140aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:06.115455 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd8779-1385-446f-b6bb-f38ea8140aa0 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:06.115744 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd8779-1385-446f-b6bb-f38ea8140aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:06.116017 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd8779-1385-446f-b6bb-f38ea8140aa0 None admin] PATH_INFO: `/v3/users/15d11995738c4ded976589903d34b89e` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:06.117157 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fd8779-1385-446f-b6bb-f38ea8140aa0 None admin] RBAC: Authorizing `identity:get_user(user_id=15d11995738c4ded976589903d34b89e)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:06.119096 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fd8779-1385-446f-b6bb-f38ea8140aa0 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:06.120706 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 100/199] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:01:05 2022] GET /identity/v3/users/15d11995738c4ded976589903d34b89e => generated 283 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:06.122202 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd8779-1385-446f-b6bb-f38ea8140aa0 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:06.185735 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 100/200] 10.222.0.98 () {60 vars in 1132 bytes} [Thu Aug 18 12:01:05 2022] POST /identity/v3/users => generated 297 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:01:06.186675 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d8815-3bc0-42f9-b800-b767cfd28bb0 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:06.228474 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:06.236121 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd8779-1385-446f-b6bb-f38ea8140aa0 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:06.278328 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ec7e2-b692-46d6-818a-7a3326639ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:06.278963 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ec7e2-b692-46d6-818a-7a3326639ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:06.279251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ec7e2-b692-46d6-818a-7a3326639ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:06.279566 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ec7e2-b692-46d6-818a-7a3326639ba9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:06.304802 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d8815-3bc0-42f9-b800-b767cfd28bb0 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:06.338880 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:06.345415 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55786e-b680-4c88-8cb3-43b35596e5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:06.346047 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55786e-b680-4c88-8cb3-43b35596e5c6 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:06.346338 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55786e-b680-4c88-8cb3-43b35596e5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:06.346601 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55786e-b680-4c88-8cb3-43b35596e5c6 None admin] PATH_INFO: `/v3/projects/9d8b38972ba64f509734ed17f30c88fd` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:06.349203 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b55786e-b680-4c88-8cb3-43b35596e5c6 None admin] RBAC: Authorizing `identity:get_project(project_id=9d8b38972ba64f509734ed17f30c88fd)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:06.350604 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b55786e-b680-4c88-8cb3-43b35596e5c6 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:06.356411 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 101/201] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:01:06 2022] GET /identity/v3/projects/9d8b38972ba64f509734ed17f30c88fd => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:06.356411 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55786e-b680-4c88-8cb3-43b35596e5c6 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:06.360673 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ec7e2-b692-46d6-818a-7a3326639ba9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:06.362162 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ec7e2-b692-46d6-818a-7a3326639ba9 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:06.484827 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:06.486263 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55786e-b680-4c88-8cb3-43b35596e5c6 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:06.493819 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 101/202] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:01:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:01:06.495168 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ec7e2-b692-46d6-818a-7a3326639ba9 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:06.529139 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596a323-1284-4930-96b6-542045a5e6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:06.529785 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596a323-1284-4930-96b6-542045a5e6e7 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:06.530049 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596a323-1284-4930-96b6-542045a5e6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:06.530329 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596a323-1284-4930-96b6-542045a5e6e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:06.604758 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:06.606962 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e596a323-1284-4930-96b6-542045a5e6e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:06.608597 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e596a323-1284-4930-96b6-542045a5e6e7 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:06.638749 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ec7e2-b692-46d6-818a-7a3326639ba9 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:06.691592 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ac17c-c1fe-4871-8d13-37899feb4075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:06.694753 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ac17c-c1fe-4871-8d13-37899feb4075 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:06.695187 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ac17c-c1fe-4871-8d13-37899feb4075 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:06.695644 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ac17c-c1fe-4871-8d13-37899feb4075 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:06.696582 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3ac17c-c1fe-4871-8d13-37899feb4075 None admin] RBAC: Adding query filter params (role.id=c53a0e13b58642f291c856efe1f0b87d, scope.project.id=9d8b38972ba64f509734ed17f30c88fd, user.id=15d11995738c4ded976589903d34b89e) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:06.697052 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3ac17c-c1fe-4871-8d13-37899feb4075 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:06.699287 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3ac17c-c1fe-4871-8d13-37899feb4075 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:06.707834 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 102/203] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:01:06 2022] GET /identity/v3/role_assignments?user.id=15d11995738c4ded976589903d34b89e&scope.project.id=9d8b38972ba64f509734ed17f30c88fd&role.id=c53a0e13b58642f291c856efe1f0b87d => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:06.709264 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ac17c-c1fe-4871-8d13-37899feb4075 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:06.745933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 102/204] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:01:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:01:06.754393 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596a323-1284-4930-96b6-542045a5e6e7 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:06.849387 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:06.919354 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:06.922888 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596a323-1284-4930-96b6-542045a5e6e7 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:06.931336 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ac17c-c1fe-4871-8d13-37899feb4075 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:06.965018 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc1206-f46f-4f76-8e8d-578953556a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:06.966068 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc1206-f46f-4f76-8e8d-578953556a31 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:06.966546 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc1206-f46f-4f76-8e8d-578953556a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:06.967014 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc1206-f46f-4f76-8e8d-578953556a31 None admin] PATH_INFO: `/v3/projects/9d8b38972ba64f509734ed17f30c88fd` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:06.968863 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc1206-f46f-4f76-8e8d-578953556a31 None admin] RBAC: Authorizing `identity:get_project(project_id=9d8b38972ba64f509734ed17f30c88fd)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:06.970533 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc1206-f46f-4f76-8e8d-578953556a31 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:06.972406 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 103/205] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:01:06 2022] GET /identity/v3/projects/9d8b38972ba64f509734ed17f30c88fd => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:01:06.984585 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc1206-f46f-4f76-8e8d-578953556a31 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:07.009643 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21324a6-2b34-4af1-acfa-f3e80ca5dab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:07.010291 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21324a6-2b34-4af1-acfa-f3e80ca5dab1 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:07.010577 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21324a6-2b34-4af1-acfa-f3e80ca5dab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:07.010861 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21324a6-2b34-4af1-acfa-f3e80ca5dab1 None admin] PATH_INFO: `/v3/projects/dede24cb1b014e9796b44d6cdac3ed69` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:07.012169 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21324a6-2b34-4af1-acfa-f3e80ca5dab1 None admin] RBAC: Authorizing `identity:get_project(project_id=dede24cb1b014e9796b44d6cdac3ed69)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:07.013649 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21324a6-2b34-4af1-acfa-f3e80ca5dab1 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:07.017569 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 103/206] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:01:06 2022] GET /identity/v3/projects/dede24cb1b014e9796b44d6cdac3ed69 => generated 298 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:07.024022 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21324a6-2b34-4af1-acfa-f3e80ca5dab1 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:07.167532 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21324a6-2b34-4af1-acfa-f3e80ca5dab1 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:07.203975 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:07.211335 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787cdd95-85a2-4c4d-a567-d78a57972d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:07.212149 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787cdd95-85a2-4c4d-a567-d78a57972d20 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:07.212441 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787cdd95-85a2-4c4d-a567-d78a57972d20 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:07.212725 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787cdd95-85a2-4c4d-a567-d78a57972d20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:07.213472 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787cdd95-85a2-4c4d-a567-d78a57972d20 None admin] RBAC: Adding query filter params (role.id=93cdab4d25f54b8ca825e7ee5a5dc2d2, scope.project.id=dede24cb1b014e9796b44d6cdac3ed69, user.id=955131e7b5f048a5be55cc222561df18) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:07.213789 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787cdd95-85a2-4c4d-a567-d78a57972d20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:07.217176 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787cdd95-85a2-4c4d-a567-d78a57972d20 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:07.224600 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 104/207] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:01:07 2022] GET /identity/v3/role_assignments?user.id=955131e7b5f048a5be55cc222561df18&scope.project.id=dede24cb1b014e9796b44d6cdac3ed69&role.id=93cdab4d25f54b8ca825e7ee5a5dc2d2 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:07.243740 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc1206-f46f-4f76-8e8d-578953556a31 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:07.292638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801b0915-93a3-456e-aea9-caa54758d3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:07.293953 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b0915-93a3-456e-aea9-caa54758d3a2 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:07.294248 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b0915-93a3-456e-aea9-caa54758d3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:07.294532 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b0915-93a3-456e-aea9-caa54758d3a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:07.295301 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801b0915-93a3-456e-aea9-caa54758d3a2 None admin] RBAC: Adding query filter params (role.id=3a4b7e53b8644e508ac440f71138be16, scope.project.id=9d8b38972ba64f509734ed17f30c88fd, user.id=15d11995738c4ded976589903d34b89e) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:07.295662 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801b0915-93a3-456e-aea9-caa54758d3a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:07.297152 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801b0915-93a3-456e-aea9-caa54758d3a2 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:07.307559 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 104/208] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:01:06 2022] GET /identity/v3/role_assignments?user.id=15d11995738c4ded976589903d34b89e&scope.project.id=9d8b38972ba64f509734ed17f30c88fd&role.id=3a4b7e53b8644e508ac440f71138be16 => generated 259 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:08.431017 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fd778-6f6e-42f0-8dd0-5e63a627d3ef None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:08.431583 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fd778-6f6e-42f0-8dd0-5e63a627d3ef None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:08.431671 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fd778-6f6e-42f0-8dd0-5e63a627d3ef None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:08.433323 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 105/209] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:08.438750 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6178fd-6987-4fae-8fb0-723c97864790 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:08.439636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6178fd-6987-4fae-8fb0-723c97864790 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:08.440116 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6178fd-6987-4fae-8fb0-723c97864790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:08.460208 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-ba6178fd-6987-4fae-8fb0-723c97864790 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:08.461814 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-ba6178fd-6987-4fae-8fb0-723c97864790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:08.490726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:08.536940 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 105/210] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 12:01:08.716970 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ea8b4-581e-42c5-a5e7-8d2333a0eaf0 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:08.717259 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ea8b4-581e-42c5-a5e7-8d2333a0eaf0 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:08.717535 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ea8b4-581e-42c5-a5e7-8d2333a0eaf0 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:08.718892 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 106/211] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 12:01:08.724347 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153acc1-0973-4e60-8cca-31ad946ebed8 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:08.724637 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153acc1-0973-4e60-8cca-31ad946ebed8 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:08.724921 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153acc1-0973-4e60-8cca-31ad946ebed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:08.744223 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-2153acc1-0973-4e60-8cca-31ad946ebed8 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:08.745431 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-2153acc1-0973-4e60-8cca-31ad946ebed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:08.818332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 106/212] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:08.821647 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ea8b4-581e-42c5-a5e7-8d2333a0eaf0 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:08.931079 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ea8b4-581e-42c5-a5e7-8d2333a0eaf0 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:08.995401 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e58ee7-c88a-4f16-b864-01ce6a779658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:08.996081 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e58ee7-c88a-4f16-b864-01ce6a779658 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:08.996368 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e58ee7-c88a-4f16-b864-01ce6a779658 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:08.996681 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e58ee7-c88a-4f16-b864-01ce6a779658 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:09.007635 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e58ee7-c88a-4f16-b864-01ce6a779658 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:09.013449 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-a3e58ee7-c88a-4f16-b864-01ce6a779658 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 12:01:09.013449 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:09.013449 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:09.013449 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:09.013449 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:09.013449 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:09.013449 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:09.013449 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:09.013449 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:09.013449 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:09.013449 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:09.013449 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:09.013449 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:09.013449 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:09.013449 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:09.013449 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:09.014353 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:09.014353 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:09.014353 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:09.014353 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:09.014353 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:09.014353 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:09.014353 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:09.014353 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:09.014353 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:09.014353 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:09.014353 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:09.014353 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:09.014353 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:09.014353 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:09.014353 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:09.014353 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:09.015134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:09.015134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:09.015134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:09.015134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:09.015134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:09.015134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:09.015134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:09.015134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:09.015134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:09.015134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:09.015134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 12:01:09.015134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:09.015134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 107/213] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:01:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:09.021504 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2153acc1-0973-4e60-8cca-31ad946ebed8 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:09.118662 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:09.127680 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e58ee7-c88a-4f16-b864-01ce6a779658 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:09.127837 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2153acc1-0973-4e60-8cca-31ad946ebed8 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:09.173012 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd976a0-53b3-4a4c-ad7d-2d63696f0a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:09.173640 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd976a0-53b3-4a4c-ad7d-2d63696f0a65 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:09.173926 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd976a0-53b3-4a4c-ad7d-2d63696f0a65 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:09.174202 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd976a0-53b3-4a4c-ad7d-2d63696f0a65 None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:09.174884 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd976a0-53b3-4a4c-ad7d-2d63696f0a65 None admin] RBAC: Adding query filter params (name=member) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:09.175192 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd976a0-53b3-4a4c-ad7d-2d63696f0a65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:09.177003 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd976a0-53b3-4a4c-ad7d-2d63696f0a65 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:09.192102 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 107/214] 10.222.0.98 () {56 vars in 1101 bytes} [Thu Aug 18 12:01:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:09.192276 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:09.194251 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd976a0-53b3-4a4c-ad7d-2d63696f0a65 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:09.244468 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e58ee7-c88a-4f16-b864-01ce6a779658 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:09.281313 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8afa112-faf0-4fa3-9170-dff660cc5bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:09.281876 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afa112-faf0-4fa3-9170-dff660cc5bfa None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:09.282158 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afa112-faf0-4fa3-9170-dff660cc5bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:09.282435 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afa112-faf0-4fa3-9170-dff660cc5bfa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:09.283626 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8afa112-faf0-4fa3-9170-dff660cc5bfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:09.285179 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8afa112-faf0-4fa3-9170-dff660cc5bfa None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:09.286754 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 108/215] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:01:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:01:09.288003 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8063333-8a2e-457a-bd5c-ad301d776673 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:09.288294 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8063333-8a2e-457a-bd5c-ad301d776673 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:09.288569 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8063333-8a2e-457a-bd5c-ad301d776673 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:09.289914 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 109/216] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:09.294542 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8063333-8a2e-457a-bd5c-ad301d776673 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:09.308763 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd976a0-53b3-4a4c-ad7d-2d63696f0a65 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:09.344371 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5521eb6-e58e-480a-9505-b6febfbd9cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:09.345151 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5521eb6-e58e-480a-9505-b6febfbd9cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:09.345436 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5521eb6-e58e-480a-9505-b6febfbd9cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:09.345713 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5521eb6-e58e-480a-9505-b6febfbd9cf6 None admin] PATH_INFO: `/v3/users/9a0ca00de2084bb2818c40132bc31936` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:09.358937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5521eb6-e58e-480a-9505-b6febfbd9cf6 None admin] RBAC: Authorizing `identity:get_user(user_id=9a0ca00de2084bb2818c40132bc31936)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:09.360432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5521eb6-e58e-480a-9505-b6febfbd9cf6 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:09.362050 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 108/217] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:01:09 2022] GET /identity/v3/users/9a0ca00de2084bb2818c40132bc31936 => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:09.363197 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5279d-c0ae-4b7d-b243-ce7ca8909ab2 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:09.363518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5279d-c0ae-4b7d-b243-ce7ca8909ab2 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:09.363798 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5279d-c0ae-4b7d-b243-ce7ca8909ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:09.381950 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-c3f5279d-c0ae-4b7d-b243-ce7ca8909ab2 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:09.383123 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-c3f5279d-c0ae-4b7d-b243-ce7ca8909ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:09.391151 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:09.403145 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8063333-8a2e-457a-bd5c-ad301d776673 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:09.422685 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:09.437549 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38970baf-e63e-4372-91a2-8e694a6a59fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:09.437549 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38970baf-e63e-4372-91a2-8e694a6a59fb None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:09.439718 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38970baf-e63e-4372-91a2-8e694a6a59fb None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:09.439718 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38970baf-e63e-4372-91a2-8e694a6a59fb None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:09.439718 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38970baf-e63e-4372-91a2-8e694a6a59fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:09.440032 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38970baf-e63e-4372-91a2-8e694a6a59fb None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:09.455926 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 109/218] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:09.457786 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31349e6-c3a7-4f4f-8076-701cb5819877 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:09.458075 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31349e6-c3a7-4f4f-8076-701cb5819877 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:09.458349 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31349e6-c3a7-4f4f-8076-701cb5819877 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:09.459624 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 110/219] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:09.463753 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31349e6-c3a7-4f4f-8076-701cb5819877 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:09.552083 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 110/220] 10.222.0.98 () {60 vars in 1132 bytes} [Thu Aug 18 12:01:09 2022] POST /identity/v3/users => generated 297 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:01:09.553573 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaee10d-06de-491d-8498-1b791216f43b None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:09.554210 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaee10d-06de-491d-8498-1b791216f43b None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:09.554210 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaee10d-06de-491d-8498-1b791216f43b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:09.584403 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31349e6-c3a7-4f4f-8076-701cb5819877 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:09.584612 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-6eaee10d-06de-491d-8498-1b791216f43b None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:09.585934 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-6eaee10d-06de-491d-8498-1b791216f43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:09.626545 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e1534-1a44-4473-a371-33e54d976f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:09.627608 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e1534-1a44-4473-a371-33e54d976f5a None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:09.628096 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e1534-1a44-4473-a371-33e54d976f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:09.628589 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e1534-1a44-4473-a371-33e54d976f5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:09.647846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:09.667049 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 111/221] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:09.724954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e1534-1a44-4473-a371-33e54d976f5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:09.726427 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e1534-1a44-4473-a371-33e54d976f5a None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:09.743063 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaee10d-06de-491d-8498-1b791216f43b None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:09.843536 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 111/222] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:01:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:01:09.849098 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e1534-1a44-4473-a371-33e54d976f5a None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:09.891875 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaee10d-06de-491d-8498-1b791216f43b None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:09.901054 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:09.909829 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:09.939080 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b0d6a-00eb-4120-846a-3bc83667e183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:09.939944 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b0d6a-00eb-4120-846a-3bc83667e183 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:09.940230 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b0d6a-00eb-4120-846a-3bc83667e183 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:09.941500 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b0d6a-00eb-4120-846a-3bc83667e183 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:09.956006 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b0d6a-00eb-4120-846a-3bc83667e183 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:09.961720 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-624b0d6a-00eb-4120-846a-3bc83667e183 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:01:09.961720 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:09.961720 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:09.961720 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:09.961720 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:09.961720 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:09.961720 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:09.961720 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:09.961720 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:09.961720 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:09.961720 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:09.961720 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:09.961720 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:09.961720 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:09.961720 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:09.961720 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:09.962657 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:09.962657 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:09.962657 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:09.962657 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:09.962657 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:09.962657 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:09.962657 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:09.962657 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:09.962657 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:09.962657 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:09.962657 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:09.962657 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:09.962657 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:09.962657 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:09.962657 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:09.962657 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:09.963751 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:09.963751 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:09.963751 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:09.963751 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:09.963751 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:09.963751 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:09.963751 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:09.963751 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:09.963751 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:09.963751 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:09.963751 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:01:09.963751 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:09.963751 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 112/223] 10.222.0.98 () {56 vars in 1096 bytes} [Thu Aug 18 12:01:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:01:09.963751 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b0d6a-00eb-4120-846a-3bc83667e183 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:09.977894 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e1534-1a44-4473-a371-33e54d976f5a None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:10.017846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8eec98-0459-4b3f-a898-e0646c75c0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:10.018540 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8eec98-0459-4b3f-a898-e0646c75c0ed None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:10.018828 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8eec98-0459-4b3f-a898-e0646c75c0ed None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:10.019105 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8eec98-0459-4b3f-a898-e0646c75c0ed None admin] PATH_INFO: `/v3/projects/9d8b38972ba64f509734ed17f30c88fd` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:10.020587 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8eec98-0459-4b3f-a898-e0646c75c0ed None admin] RBAC: Authorizing `identity:get_project(project_id=9d8b38972ba64f509734ed17f30c88fd)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:10.022132 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8eec98-0459-4b3f-a898-e0646c75c0ed None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:10.023856 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 112/224] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:01:09 2022] GET /identity/v3/projects/9d8b38972ba64f509734ed17f30c88fd => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:10.024634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8eec98-0459-4b3f-a898-e0646c75c0ed None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:10.088728 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b0d6a-00eb-4120-846a-3bc83667e183 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:10.142110 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61334835-f9da-4514-a9e3-e6a24ceba6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:10.143023 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61334835-f9da-4514-a9e3-e6a24ceba6a3 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:10.143714 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61334835-f9da-4514-a9e3-e6a24ceba6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:10.143996 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61334835-f9da-4514-a9e3-e6a24ceba6a3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:10.149351 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8eec98-0459-4b3f-a898-e0646c75c0ed None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:10.157978 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61334835-f9da-4514-a9e3-e6a24ceba6a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:10.163082 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:10.166062 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-61334835-f9da-4514-a9e3-e6a24ceba6a3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 12:01:10.166062 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:10.166062 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:10.166062 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:10.166062 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:10.166062 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:10.166062 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:10.166062 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:10.166062 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:10.166062 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:10.166062 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:10.166062 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:10.166062 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:10.166062 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:10.166062 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:10.166062 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:10.167155 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:10.167155 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:10.167155 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:10.167155 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:10.167155 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:10.167155 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:10.167155 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:10.167155 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:10.167155 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:10.167155 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:10.167155 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:10.167155 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:10.167155 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:10.167155 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:10.167155 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:10.167155 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:10.168221 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:10.168221 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:10.168221 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:10.168221 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:10.168221 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:10.168221 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:10.168221 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:10.168221 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:10.168221 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:10.168221 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:10.168221 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 12:01:10.168221 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:10.171011 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 113/225] 10.222.0.98 () {56 vars in 1114 bytes} [Thu Aug 18 12:01:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:10.176576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61334835-f9da-4514-a9e3-e6a24ceba6a3 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:10.205329 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afd8ab8-c609-4a61-af79-880a71064353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:10.206826 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd8ab8-c609-4a61-af79-880a71064353 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:10.207260 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd8ab8-c609-4a61-af79-880a71064353 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:10.207715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd8ab8-c609-4a61-af79-880a71064353 None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:10.208812 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afd8ab8-c609-4a61-af79-880a71064353 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:10.209281 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afd8ab8-c609-4a61-af79-880a71064353 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:10.211883 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afd8ab8-c609-4a61-af79-880a71064353 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:10.214744 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:10.230983 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 113/226] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:01:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:10.232210 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afd8ab8-c609-4a61-af79-880a71064353 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:10.310498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61334835-f9da-4514-a9e3-e6a24ceba6a3 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:10.356167 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4f124-5126-4a6b-8fad-c0782d88f904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:10.357035 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4f124-5126-4a6b-8fad-c0782d88f904 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:10.357414 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4f124-5126-4a6b-8fad-c0782d88f904 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:10.357788 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4f124-5126-4a6b-8fad-c0782d88f904 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:10.358654 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4f124-5126-4a6b-8fad-c0782d88f904 None admin] RBAC: Adding query filter params (role.id=c81e28908b9b49a38d33978df4d11716, scope.project.id=9d8b38972ba64f509734ed17f30c88fd, user.id=9a0ca00de2084bb2818c40132bc31936) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:10.358944 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afd8ab8-c609-4a61-af79-880a71064353 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:10.359152 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4f124-5126-4a6b-8fad-c0782d88f904 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:10.361016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4f124-5126-4a6b-8fad-c0782d88f904 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:10.368099 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 114/227] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:01:10 2022] GET /identity/v3/role_assignments?user.id=9a0ca00de2084bb2818c40132bc31936&scope.project.id=9d8b38972ba64f509734ed17f30c88fd&role.id=c81e28908b9b49a38d33978df4d11716 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:10.370931 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4f124-5126-4a6b-8fad-c0782d88f904 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:10.414529 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b560f69-366a-4e6c-b9e4-8d70ee5ff597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:10.418037 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b560f69-366a-4e6c-b9e4-8d70ee5ff597 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:10.418327 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b560f69-366a-4e6c-b9e4-8d70ee5ff597 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:10.419152 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b560f69-366a-4e6c-b9e4-8d70ee5ff597 None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:10.419871 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b560f69-366a-4e6c-b9e4-8d70ee5ff597 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:10.420184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b560f69-366a-4e6c-b9e4-8d70ee5ff597 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:10.424342 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b560f69-366a-4e6c-b9e4-8d70ee5ff597 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:10.449926 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 114/228] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:01:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:10.451001 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b560f69-366a-4e6c-b9e4-8d70ee5ff597 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:10.520695 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:10.561289 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:10.577815 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4f124-5126-4a6b-8fad-c0782d88f904 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:10.626935 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53c1cd-2ab4-4072-b3f7-23c0c1aacb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:10.628653 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53c1cd-2ab4-4072-b3f7-23c0c1aacb58 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:10.630268 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53c1cd-2ab4-4072-b3f7-23c0c1aacb58 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:10.630963 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53c1cd-2ab4-4072-b3f7-23c0c1aacb58 None admin] PATH_INFO: `/v3/users/15d11995738c4ded976589903d34b89e` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:10.633324 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a53c1cd-2ab4-4072-b3f7-23c0c1aacb58 None admin] RBAC: Authorizing `identity:get_user(user_id=15d11995738c4ded976589903d34b89e)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:10.635746 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a53c1cd-2ab4-4072-b3f7-23c0c1aacb58 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:10.639151 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b560f69-366a-4e6c-b9e4-8d70ee5ff597 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:10.641025 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 115/229] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:01:10 2022] GET /identity/v3/users/15d11995738c4ded976589903d34b89e => generated 283 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:10.650113 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53c1cd-2ab4-4072-b3f7-23c0c1aacb58 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:10.702494 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b16bf3-c3d5-4f03-8986-8d381789441e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:10.703140 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b16bf3-c3d5-4f03-8986-8d381789441e None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:10.703445 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b16bf3-c3d5-4f03-8986-8d381789441e None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:10.703707 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b16bf3-c3d5-4f03-8986-8d381789441e None admin] PATH_INFO: `/v3/users/15d11995738c4ded976589903d34b89e` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:10.704903 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b16bf3-c3d5-4f03-8986-8d381789441e None admin] RBAC: Authorizing `identity:get_user(user_id=15d11995738c4ded976589903d34b89e)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:10.706368 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b16bf3-c3d5-4f03-8986-8d381789441e None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:10.712678 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 115/230] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:01:10 2022] GET /identity/v3/users/15d11995738c4ded976589903d34b89e => generated 283 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:10.713561 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b16bf3-c3d5-4f03-8986-8d381789441e None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:10.864467 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b16bf3-c3d5-4f03-8986-8d381789441e None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:10.865999 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:10.897622 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53c1cd-2ab4-4072-b3f7-23c0c1aacb58 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:10.907777 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb49b49-e5bc-42fe-9c76-b0367e30f2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:10.908436 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb49b49-e5bc-42fe-9c76-b0367e30f2ee None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:10.908719 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb49b49-e5bc-42fe-9c76-b0367e30f2ee None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:10.908997 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb49b49-e5bc-42fe-9c76-b0367e30f2ee None admin] PATH_INFO: `/v3/projects/9d8b38972ba64f509734ed17f30c88fd` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:10.910211 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb49b49-e5bc-42fe-9c76-b0367e30f2ee None admin] RBAC: Authorizing `identity:get_project(project_id=9d8b38972ba64f509734ed17f30c88fd)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:10.911715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb49b49-e5bc-42fe-9c76-b0367e30f2ee None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:10.913702 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 116/231] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:01:10 2022] GET /identity/v3/projects/9d8b38972ba64f509734ed17f30c88fd => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:10.921811 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb49b49-e5bc-42fe-9c76-b0367e30f2ee None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:10.971345 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89cbb0-7e53-4eff-82b8-90b43cbab72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:10.972055 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89cbb0-7e53-4eff-82b8-90b43cbab72c None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:10.972344 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89cbb0-7e53-4eff-82b8-90b43cbab72c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:10.972624 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89cbb0-7e53-4eff-82b8-90b43cbab72c None admin] PATH_INFO: `/v3/projects/9d8b38972ba64f509734ed17f30c88fd` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:10.973910 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df89cbb0-7e53-4eff-82b8-90b43cbab72c None admin] RBAC: Authorizing `identity:get_project(project_id=9d8b38972ba64f509734ed17f30c88fd)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:10.975519 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df89cbb0-7e53-4eff-82b8-90b43cbab72c None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:10.977333 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 116/232] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:01:10 2022] GET /identity/v3/projects/9d8b38972ba64f509734ed17f30c88fd => generated 302 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:10.983716 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89cbb0-7e53-4eff-82b8-90b43cbab72c None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:11.012055 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:11.082196 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb49b49-e5bc-42fe-9c76-b0367e30f2ee None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:11.089820 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89cbb0-7e53-4eff-82b8-90b43cbab72c None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:11.126408 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc504c-6b88-463e-8847-222cee8b3a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:11.127082 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc504c-6b88-463e-8847-222cee8b3a9a None admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:11.127366 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc504c-6b88-463e-8847-222cee8b3a9a None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:11.127671 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc504c-6b88-463e-8847-222cee8b3a9a None admin] PATH_INFO: `/v3/projects/9d8b38972ba64f509734ed17f30c88fd/users/15d11995738c4ded976589903d34b89e/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:11.129597 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fc504c-6b88-463e-8847-222cee8b3a9a None admin] RBAC: Authorizing `identity:create_grant(project_id=9d8b38972ba64f509734ed17f30c88fd, user_id=15d11995738c4ded976589903d34b89e, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:11.135438 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fc504c-6b88-463e-8847-222cee8b3a9a None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:11.150891 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9666abe2-3aad-4f6d-bbc6-3a596fdcf200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:11.151587 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666abe2-3aad-4f6d-bbc6-3a596fdcf200 None admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:11.151868 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666abe2-3aad-4f6d-bbc6-3a596fdcf200 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:11.152143 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666abe2-3aad-4f6d-bbc6-3a596fdcf200 None admin] PATH_INFO: `/v3/projects/9d8b38972ba64f509734ed17f30c88fd/users/15d11995738c4ded976589903d34b89e/roles/3a4b7e53b8644e508ac440f71138be16` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:11.154365 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9666abe2-3aad-4f6d-bbc6-3a596fdcf200 None admin] RBAC: Authorizing `identity:create_grant(project_id=9d8b38972ba64f509734ed17f30c88fd, user_id=15d11995738c4ded976589903d34b89e, role_id=3a4b7e53b8644e508ac440f71138be16)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:11.155938 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9666abe2-3aad-4f6d-bbc6-3a596fdcf200 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:11.232520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 117/233] 10.222.0.98 () {58 vars in 1439 bytes} [Thu Aug 18 12:01:10 2022] PUT /identity/v3/projects/9d8b38972ba64f509734ed17f30c88fd/users/15d11995738c4ded976589903d34b89e/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:01:11.272592 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 117/234] 10.222.0.98 () {58 vars in 1439 bytes} [Thu Aug 18 12:01:10 2022] PUT /identity/v3/projects/9d8b38972ba64f509734ed17f30c88fd/users/15d11995738c4ded976589903d34b89e/roles/3a4b7e53b8644e508ac440f71138be16 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:01:11.932296 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f314fa-6535-4f21-b4b2-64d7d2dd6fc7 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:11.932809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f314fa-6535-4f21-b4b2-64d7d2dd6fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:11.932883 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f314fa-6535-4f21-b4b2-64d7d2dd6fc7 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:11.935058 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 118/235] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:11.940649 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0e6bf-2234-4291-9f2b-b16ef02b14dc None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:11.941099 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0e6bf-2234-4291-9f2b-b16ef02b14dc None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:11.941277 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0e6bf-2234-4291-9f2b-b16ef02b14dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:11.968458 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-9fd0e6bf-2234-4291-9f2b-b16ef02b14dc None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:11.969703 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-9fd0e6bf-2234-4291-9f2b-b16ef02b14dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:12.005287 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:12.065709 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 118/236] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:12.154750 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b8d9c-dabd-4ef2-9dc9-84f2a27052c0 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:12.155066 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b8d9c-dabd-4ef2-9dc9-84f2a27052c0 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:12.155320 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b8d9c-dabd-4ef2-9dc9-84f2a27052c0 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:12.156987 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 119/237] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:12.162301 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448b6ea-269d-45a9-91fd-e7d56cb85fc0 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:12.162594 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448b6ea-269d-45a9-91fd-e7d56cb85fc0 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:12.162873 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448b6ea-269d-45a9-91fd-e7d56cb85fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:12.192424 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-f448b6ea-269d-45a9-91fd-e7d56cb85fc0 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:12.193638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-f448b6ea-269d-45a9-91fd-e7d56cb85fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:12.277350 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 119/238] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:12.500470 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b8d9c-dabd-4ef2-9dc9-84f2a27052c0 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:12.594001 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:12.612311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b8d9c-dabd-4ef2-9dc9-84f2a27052c0 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:12.619240 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045f348-7553-460c-9fe9-d836c8ef7487 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:12.619562 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045f348-7553-460c-9fe9-d836c8ef7487 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:12.619845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045f348-7553-460c-9fe9-d836c8ef7487 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:12.621193 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 120/239] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:12.627374 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e3548-668b-45d3-9302-8459d2405055 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:12.627691 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e3548-668b-45d3-9302-8459d2405055 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:12.627970 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e3548-668b-45d3-9302-8459d2405055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:12.649532 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609dc11-c317-4391-97d5-fe9565b7390a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:12.650823 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609dc11-c317-4391-97d5-fe9565b7390a None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:12.651312 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609dc11-c317-4391-97d5-fe9565b7390a None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:12.651848 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609dc11-c317-4391-97d5-fe9565b7390a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:12.663945 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-4b1e3548-668b-45d3-9302-8459d2405055 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:12.667820 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-4b1e3548-668b-45d3-9302-8459d2405055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:12.674600 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4609dc11-c317-4391-97d5-fe9565b7390a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:12.684251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-4609dc11-c317-4391-97d5-fe9565b7390a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 12:01:12.684251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:12.684251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:12.684251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:12.684251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:12.684251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:12.684251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:12.684251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:12.684251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:12.684251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:12.684251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:12.684251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:12.684251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:12.684251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:12.684251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:12.684251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:12.685133 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:12.685133 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:12.685133 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:12.685133 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:12.685133 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:12.685133 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:12.685133 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:12.685133 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:12.685133 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:12.685133 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:12.685133 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:12.685133 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:12.685133 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:12.685133 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:12.685133 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:12.685133 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:12.685921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:12.685921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:12.685921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:12.685921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:12.685921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:12.685921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:12.685921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:12.685921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:12.685921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:12.685921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:12.685921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 12:01:12.685921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:12.688579 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 120/240] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:01:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:12.695207 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609dc11-c317-4391-97d5-fe9565b7390a None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:12.812252 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 121/241] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 12:01:12.813474 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1e3548-668b-45d3-9302-8459d2405055 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:12.830737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609dc11-c317-4391-97d5-fe9565b7390a None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:12.863462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:12.907749 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de20df3-b3e1-46f8-a352-059d3a88f69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:12.909757 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de20df3-b3e1-46f8-a352-059d3a88f69d None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:12.911740 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de20df3-b3e1-46f8-a352-059d3a88f69d None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:12.912234 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de20df3-b3e1-46f8-a352-059d3a88f69d None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:12.913121 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de20df3-b3e1-46f8-a352-059d3a88f69d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:12.914462 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de20df3-b3e1-46f8-a352-059d3a88f69d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:12.916144 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de20df3-b3e1-46f8-a352-059d3a88f69d None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:12.934811 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 121/242] 10.222.0.98 () {56 vars in 1101 bytes} [Thu Aug 18 12:01:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:12.943208 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de20df3-b3e1-46f8-a352-059d3a88f69d None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:12.955987 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:13.036497 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1e3548-668b-45d3-9302-8459d2405055 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:13.084868 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de20df3-b3e1-46f8-a352-059d3a88f69d None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:13.109157 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e497c-a39d-474e-84df-2cdd11f375c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:13.109810 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e497c-a39d-474e-84df-2cdd11f375c2 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:13.110095 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e497c-a39d-474e-84df-2cdd11f375c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:13.110373 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e497c-a39d-474e-84df-2cdd11f375c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:13.116341 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342e497c-a39d-474e-84df-2cdd11f375c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:13.117940 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342e497c-a39d-474e-84df-2cdd11f375c2 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:13.119607 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 122/243] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:01:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:13.122902 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06b0d90-0557-4ea8-afb0-3a2c2654c16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:13.123577 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b0d90-0557-4ea8-afb0-3a2c2654c16b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:13.123921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b0d90-0557-4ea8-afb0-3a2c2654c16b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:13.124200 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b0d90-0557-4ea8-afb0-3a2c2654c16b None admin] PATH_INFO: `/v3/users/0fc76e658e5046719fd84efc3749bdce` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:13.125717 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e497c-a39d-474e-84df-2cdd11f375c2 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:13.139691 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06b0d90-0557-4ea8-afb0-3a2c2654c16b None admin] RBAC: Authorizing `identity:get_user(user_id=0fc76e658e5046719fd84efc3749bdce)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:13.141176 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06b0d90-0557-4ea8-afb0-3a2c2654c16b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:13.143656 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 122/244] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:01:12 2022] GET /identity/v3/users/0fc76e658e5046719fd84efc3749bdce => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:13.149859 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06b0d90-0557-4ea8-afb0-3a2c2654c16b None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:13.220653 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:13.241748 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:13.253987 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e497c-a39d-474e-84df-2cdd11f375c2 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:13.259834 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06b0d90-0557-4ea8-afb0-3a2c2654c16b None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:13.292008 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a1453-78ce-4284-abaf-c669f512cd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:13.292628 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a1453-78ce-4284-abaf-c669f512cd48 None admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:13.292960 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a1453-78ce-4284-abaf-c669f512cd48 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:13.293240 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a1453-78ce-4284-abaf-c669f512cd48 None admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:13.294486 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a1453-78ce-4284-abaf-c669f512cd48 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:13.296066 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a1453-78ce-4284-abaf-c669f512cd48 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:13.297264 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d09edf-c2b0-4186-b337-52a49b4f81e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:13.297946 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d09edf-c2b0-4186-b337-52a49b4f81e4 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:13.298291 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d09edf-c2b0-4186-b337-52a49b4f81e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:13.299168 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d09edf-c2b0-4186-b337-52a49b4f81e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:13.372604 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d09edf-c2b0-4186-b337-52a49b4f81e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:13.373973 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d09edf-c2b0-4186-b337-52a49b4f81e4 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:13.388230 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 123/245] 10.222.0.98 () {60 vars in 1132 bytes} [Thu Aug 18 12:01:13 2022] POST /identity/v3/users => generated 293 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:01:13.389610 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a1453-78ce-4284-abaf-c669f512cd48 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:13.451207 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:13.493971 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 123/246] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:01:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:01:13.499933 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d09edf-c2b0-4186-b337-52a49b4f81e4 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:13.538817 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a1453-78ce-4284-abaf-c669f512cd48 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:13.581632 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d23744-0c47-4fc1-b358-09a8809365bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:13.582562 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d23744-0c47-4fc1-b358-09a8809365bf None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:13.582913 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d23744-0c47-4fc1-b358-09a8809365bf None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:13.583269 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d23744-0c47-4fc1-b358-09a8809365bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:13.600738 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d23744-0c47-4fc1-b358-09a8809365bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:13.606031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-14d23744-0c47-4fc1-b358-09a8809365bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 12:01:13.606031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:13.606031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:13.606031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:13.606031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:13.606031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:13.606031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:13.606031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:13.606031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:13.606031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:13.606031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:13.606031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:13.606031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:13.606031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:13.606031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:13.606031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:13.607092 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:13.607092 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:13.607092 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:13.607092 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:13.607092 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:13.607092 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:13.607092 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:13.607092 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:13.607092 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:13.607092 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:13.607092 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:13.607092 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:13.607092 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:13.607092 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:13.607092 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:13.607092 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:13.607905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:13.607905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:13.607905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:13.607905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:13.607905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:13.607905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:13.607905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:13.607905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:13.607905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:13.607905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:13.607905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 12:01:13.607905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:13.607905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 124/247] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:01:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:13.609011 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b9e4a-d3b2-4bee-912c-beb6f8249397 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:13.609011 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b9e4a-d3b2-4bee-912c-beb6f8249397 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:13.609276 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b9e4a-d3b2-4bee-912c-beb6f8249397 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:13.610762 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 125/248] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:13.614603 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b9e4a-d3b2-4bee-912c-beb6f8249397 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:13.618612 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:13.633805 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d09edf-c2b0-4186-b337-52a49b4f81e4 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:13.681134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5b9b2d-f081-4a2d-80eb-eff1883d0209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:13.682042 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b9b2d-f081-4a2d-80eb-eff1883d0209 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:13.682378 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b9b2d-f081-4a2d-80eb-eff1883d0209 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:13.682665 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b9b2d-f081-4a2d-80eb-eff1883d0209 None admin] PATH_INFO: `/v3/projects/9d8b38972ba64f509734ed17f30c88fd` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:13.683883 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5b9b2d-f081-4a2d-80eb-eff1883d0209 None admin] RBAC: Authorizing `identity:get_project(project_id=9d8b38972ba64f509734ed17f30c88fd)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:13.685454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5b9b2d-f081-4a2d-80eb-eff1883d0209 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:13.687129 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 124/249] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:01:13 2022] GET /identity/v3/projects/9d8b38972ba64f509734ed17f30c88fd => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:01:13.688501 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2fd83-a395-46ef-80bd-47b8da853f7b None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:13.688792 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2fd83-a395-46ef-80bd-47b8da853f7b None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:13.689070 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2fd83-a395-46ef-80bd-47b8da853f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:13.714271 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-9df2fd83-a395-46ef-80bd-47b8da853f7b None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:13.716318 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-9df2fd83-a395-46ef-80bd-47b8da853f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:13.756743 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:13.761293 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b9e4a-d3b2-4bee-912c-beb6f8249397 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:13.803987 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 125/250] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:13.804715 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2fd83-a395-46ef-80bd-47b8da853f7b None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:13.809917 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1facf220-5a62-4767-942c-112c03cbb965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:13.810543 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1facf220-5a62-4767-942c-112c03cbb965 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:13.810829 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1facf220-5a62-4767-942c-112c03cbb965 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:13.811104 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1facf220-5a62-4767-942c-112c03cbb965 None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:13.811820 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1facf220-5a62-4767-942c-112c03cbb965 None admin] RBAC: Adding query filter params (name=member) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:13.812191 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1facf220-5a62-4767-942c-112c03cbb965 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:13.813638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1facf220-5a62-4767-942c-112c03cbb965 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:13.827362 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 126/251] 10.222.0.98 () {56 vars in 1101 bytes} [Thu Aug 18 12:01:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:01:13.829116 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ecb41-7591-401d-8103-4f828451f568 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:13.830201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ecb41-7591-401d-8103-4f828451f568 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:13.830851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ecb41-7591-401d-8103-4f828451f568 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:13.837117 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 127/252] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:13.837117 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ecb41-7591-401d-8103-4f828451f568 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:13.934018 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:13.940418 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2fd83-a395-46ef-80bd-47b8da853f7b None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:13.983293 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ecb41-7591-401d-8103-4f828451f568 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:13.993979 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6b7e4-2015-4904-ad8a-cd3554cd6717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:13.995327 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6b7e4-2015-4904-ad8a-cd3554cd6717 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:13.995890 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6b7e4-2015-4904-ad8a-cd3554cd6717 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:13.996618 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6b7e4-2015-4904-ad8a-cd3554cd6717 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:13.998095 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee6b7e4-2015-4904-ad8a-cd3554cd6717 None admin] RBAC: Adding query filter params (role.id=93cdab4d25f54b8ca825e7ee5a5dc2d2, scope.project.id=9d8b38972ba64f509734ed17f30c88fd, user.id=0fc76e658e5046719fd84efc3749bdce) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:13.998729 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee6b7e4-2015-4904-ad8a-cd3554cd6717 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:14.001096 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee6b7e4-2015-4904-ad8a-cd3554cd6717 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:14.009493 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 126/253] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:01:13 2022] GET /identity/v3/role_assignments?user.id=0fc76e658e5046719fd84efc3749bdce&scope.project.id=9d8b38972ba64f509734ed17f30c88fd&role.id=93cdab4d25f54b8ca825e7ee5a5dc2d2 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:14.011269 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3a20c-70c0-47a6-95ff-63aa74106cd1 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:14.011946 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3a20c-70c0-47a6-95ff-63aa74106cd1 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:14.012464 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3a20c-70c0-47a6-95ff-63aa74106cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:14.021922 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee4574-2153-46fe-bdc9-73d24100717f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:14.022755 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee4574-2153-46fe-bdc9-73d24100717f None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:14.023151 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee4574-2153-46fe-bdc9-73d24100717f None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:14.023553 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee4574-2153-46fe-bdc9-73d24100717f None admin] PATH_INFO: `/v3/users/9a0ca00de2084bb2818c40132bc31936` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:14.024978 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee4574-2153-46fe-bdc9-73d24100717f None admin] RBAC: Authorizing `identity:get_user(user_id=9a0ca00de2084bb2818c40132bc31936)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:14.026528 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee4574-2153-46fe-bdc9-73d24100717f None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:14.028588 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 128/254] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:01:13 2022] GET /identity/v3/users/9a0ca00de2084bb2818c40132bc31936 => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:14.036491 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee4574-2153-46fe-bdc9-73d24100717f None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:14.036972 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-b3c3a20c-70c0-47a6-95ff-63aa74106cd1 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:14.038424 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-b3c3a20c-70c0-47a6-95ff-63aa74106cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:14.070792 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:14.130551 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 127/255] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:14.155448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3a20c-70c0-47a6-95ff-63aa74106cd1 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:14.266218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee4574-2153-46fe-bdc9-73d24100717f None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:14.281854 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3a20c-70c0-47a6-95ff-63aa74106cd1 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:14.322976 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eca360-fb81-41f0-a988-f261f255d605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:14.323901 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eca360-fb81-41f0-a988-f261f255d605 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:14.324335 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eca360-fb81-41f0-a988-f261f255d605 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:14.324760 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eca360-fb81-41f0-a988-f261f255d605 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:14.337858 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eca360-fb81-41f0-a988-f261f255d605 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:14.343563 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-a6eca360-fb81-41f0-a988-f261f255d605 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:01:14.343563 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:14.343563 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:14.343563 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:14.343563 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:14.343563 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:14.343563 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:14.343563 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:14.343563 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:14.343563 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:14.343563 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:14.343563 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:14.343563 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:14.343563 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:14.343563 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:14.343563 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:14.344641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:14.344641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:14.344641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:14.344641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:14.344641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:14.344641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:14.344641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:14.344641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:14.344641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:14.344641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:14.344641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:14.344641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:14.344641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:14.344641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:14.344641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:14.344641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:14.345580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:14.345580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:14.345580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:14.345580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:14.345580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:14.345580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:14.345580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:14.345580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:14.345580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:14.345580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:14.345580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:01:14.345580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:14.348255 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 128/256] 10.222.0.98 () {56 vars in 1096 bytes} [Thu Aug 18 12:01:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:14.357010 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eca360-fb81-41f0-a988-f261f255d605 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:14.360122 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9d393c-0c64-43a9-9a51-61d843235612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:14.360778 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d393c-0c64-43a9-9a51-61d843235612 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:14.361179 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d393c-0c64-43a9-9a51-61d843235612 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:14.361327 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d393c-0c64-43a9-9a51-61d843235612 None admin] PATH_INFO: `/v3/projects/9d8b38972ba64f509734ed17f30c88fd` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:14.375440 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9d393c-0c64-43a9-9a51-61d843235612 None admin] RBAC: Authorizing `identity:get_project(project_id=9d8b38972ba64f509734ed17f30c88fd)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:14.375440 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9d393c-0c64-43a9-9a51-61d843235612 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:14.375440 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 129/257] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:01:14 2022] GET /identity/v3/projects/9d8b38972ba64f509734ed17f30c88fd => generated 302 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:14.380323 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9d393c-0c64-43a9-9a51-61d843235612 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:14.385494 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:14.387683 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:14.533481 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eca360-fb81-41f0-a988-f261f255d605 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:14.566064 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9d393c-0c64-43a9-9a51-61d843235612 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:14.601464 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae23b9-13b6-4640-84a5-9fb5fea99f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:14.602079 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae23b9-13b6-4640-84a5-9fb5fea99f1c None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:14.602358 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae23b9-13b6-4640-84a5-9fb5fea99f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:14.602629 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae23b9-13b6-4640-84a5-9fb5fea99f1c None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:14.603304 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ae23b9-13b6-4640-84a5-9fb5fea99f1c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:14.603628 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ae23b9-13b6-4640-84a5-9fb5fea99f1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:14.605000 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ae23b9-13b6-4640-84a5-9fb5fea99f1c None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:14.622526 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 129/258] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:01:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:14.623401 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae23b9-13b6-4640-84a5-9fb5fea99f1c None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:14.626628 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c6c0d-39d9-4216-9470-a681cafdeb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:14.627300 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c6c0d-39d9-4216-9470-a681cafdeb20 None admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:14.627616 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c6c0d-39d9-4216-9470-a681cafdeb20 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:14.627893 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c6c0d-39d9-4216-9470-a681cafdeb20 None admin] PATH_INFO: `/v3/projects/9d8b38972ba64f509734ed17f30c88fd/users/9a0ca00de2084bb2818c40132bc31936/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:14.631947 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30c6c0d-39d9-4216-9470-a681cafdeb20 None admin] RBAC: Authorizing `identity:create_grant(project_id=9d8b38972ba64f509734ed17f30c88fd, user_id=9a0ca00de2084bb2818c40132bc31936, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:14.633396 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30c6c0d-39d9-4216-9470-a681cafdeb20 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:14.720174 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:14.723431 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 130/259] 10.222.0.98 () {58 vars in 1439 bytes} [Thu Aug 18 12:01:14 2022] PUT /identity/v3/projects/9d8b38972ba64f509734ed17f30c88fd/users/9a0ca00de2084bb2818c40132bc31936/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:01:14.724387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c6c0d-39d9-4216-9470-a681cafdeb20 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:14.742222 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae23b9-13b6-4640-84a5-9fb5fea99f1c None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:14.779623 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7071a035-5463-414c-b396-c05103ea957a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:14.780288 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071a035-5463-414c-b396-c05103ea957a None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:14.780574 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071a035-5463-414c-b396-c05103ea957a None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:14.780854 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071a035-5463-414c-b396-c05103ea957a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:14.795926 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7071a035-5463-414c-b396-c05103ea957a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:14.800881 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-7071a035-5463-414c-b396-c05103ea957a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 12:01:14.800881 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:14.800881 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:14.800881 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:14.800881 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:14.800881 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:14.800881 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:14.800881 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:14.800881 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:14.800881 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:14.800881 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:14.800881 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:14.800881 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:14.800881 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:14.800881 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:14.800881 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:14.801765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:14.801765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:14.801765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:14.801765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:14.801765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:14.801765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:14.801765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:14.801765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:14.801765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:14.801765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:14.801765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:14.801765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:14.801765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:14.801765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:14.801765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:14.801765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:14.802644 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:14.802644 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:14.802644 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:14.802644 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:14.802644 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:14.802644 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:14.802644 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:14.802644 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:14.802644 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:14.802644 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:14.802644 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 12:01:14.802644 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:14.802644 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 130/260] 10.222.0.98 () {56 vars in 1114 bytes} [Thu Aug 18 12:01:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:14.808781 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7071a035-5463-414c-b396-c05103ea957a None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:14.884126 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:14.933105 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7071a035-5463-414c-b396-c05103ea957a None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:14.968471 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c6c0d-39d9-4216-9470-a681cafdeb20 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:14.970841 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19fcea-53a3-4a47-8502-8bf8d326d17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:14.971505 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19fcea-53a3-4a47-8502-8bf8d326d17c None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:14.971799 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19fcea-53a3-4a47-8502-8bf8d326d17c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:14.972080 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19fcea-53a3-4a47-8502-8bf8d326d17c None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:14.972759 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a19fcea-53a3-4a47-8502-8bf8d326d17c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:14.973071 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a19fcea-53a3-4a47-8502-8bf8d326d17c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:14.974470 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a19fcea-53a3-4a47-8502-8bf8d326d17c None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:14.985825 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 131/261] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:01:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:14.991093 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19fcea-53a3-4a47-8502-8bf8d326d17c None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:14.995372 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:15.032153 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb0215-295a-401b-9eaf-dcae654d117b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:15.032814 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb0215-295a-401b-9eaf-dcae654d117b None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:15.033100 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb0215-295a-401b-9eaf-dcae654d117b None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:15.033377 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb0215-295a-401b-9eaf-dcae654d117b None admin] PATH_INFO: `/v3/users/15d11995738c4ded976589903d34b89e` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:15.034612 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb0215-295a-401b-9eaf-dcae654d117b None admin] RBAC: Authorizing `identity:get_user(user_id=15d11995738c4ded976589903d34b89e)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:15.036171 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb0215-295a-401b-9eaf-dcae654d117b None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:15.037939 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 131/262] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:01:14 2022] GET /identity/v3/users/15d11995738c4ded976589903d34b89e => generated 283 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:15.060567 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb0215-295a-401b-9eaf-dcae654d117b None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:15.114856 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19fcea-53a3-4a47-8502-8bf8d326d17c None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:15.185883 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea2c9a-feef-460b-9be8-3ec28b4cfd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:15.186828 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea2c9a-feef-460b-9be8-3ec28b4cfd4b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:15.187294 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea2c9a-feef-460b-9be8-3ec28b4cfd4b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:15.187769 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea2c9a-feef-460b-9be8-3ec28b4cfd4b None admin] PATH_INFO: `/v3/users/15d11995738c4ded976589903d34b89e` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:15.189973 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ea2c9a-feef-460b-9be8-3ec28b4cfd4b None admin] RBAC: Authorizing `identity:get_user(user_id=15d11995738c4ded976589903d34b89e)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:15.191626 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ea2c9a-feef-460b-9be8-3ec28b4cfd4b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:15.195318 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 132/263] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:01:14 2022] GET /identity/v3/users/15d11995738c4ded976589903d34b89e => generated 283 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:15.200941 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea2c9a-feef-460b-9be8-3ec28b4cfd4b None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:15.282795 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb0215-295a-401b-9eaf-dcae654d117b None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:15.343488 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:15.349284 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:15.350074 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33399445-47a5-43d9-9baa-699c580471f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:15.351094 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33399445-47a5-43d9-9baa-699c580471f7 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:15.351094 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33399445-47a5-43d9-9baa-699c580471f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:15.351264 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33399445-47a5-43d9-9baa-699c580471f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:15.422340 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea2c9a-feef-460b-9be8-3ec28b4cfd4b None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:15.429700 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33399445-47a5-43d9-9baa-699c580471f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:15.431185 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33399445-47a5-43d9-9baa-699c580471f7 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:15.521502 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347caa9c-1a22-4075-8fbf-35839f658b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:15.522142 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347caa9c-1a22-4075-8fbf-35839f658b88 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:15.522429 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347caa9c-1a22-4075-8fbf-35839f658b88 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:15.522707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347caa9c-1a22-4075-8fbf-35839f658b88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:15.536467 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 132/264] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:01:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:01:15.542331 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33399445-47a5-43d9-9baa-699c580471f7 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:15.589562 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:15.677315 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347caa9c-1a22-4075-8fbf-35839f658b88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:15.678711 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347caa9c-1a22-4075-8fbf-35839f658b88 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:15.753363 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33399445-47a5-43d9-9baa-699c580471f7 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:15.805145 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:15.852184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f952e-f5a3-495b-8ff4-e60a219f38f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:15.852836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f952e-f5a3-495b-8ff4-e60a219f38f0 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:15.853125 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f952e-f5a3-495b-8ff4-e60a219f38f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:15.853400 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f952e-f5a3-495b-8ff4-e60a219f38f0 None admin] PATH_INFO: `/v3/projects/9d8b38972ba64f509734ed17f30c88fd` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:15.860191 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f952e-f5a3-495b-8ff4-e60a219f38f0 None admin] RBAC: Authorizing `identity:get_project(project_id=9d8b38972ba64f509734ed17f30c88fd)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:15.861683 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f952e-f5a3-495b-8ff4-e60a219f38f0 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:15.862508 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 133/265] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:01:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 662 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:01:15.863843 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 133/266] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:01:15 2022] GET /identity/v3/projects/9d8b38972ba64f509734ed17f30c88fd => generated 302 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:01:15.867893 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347caa9c-1a22-4075-8fbf-35839f658b88 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:15.869649 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f952e-f5a3-495b-8ff4-e60a219f38f0 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:16.043451 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347caa9c-1a22-4075-8fbf-35839f658b88 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:16.086281 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd82acd-5180-43d8-8983-0505b7c61318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:16.086962 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd82acd-5180-43d8-8983-0505b7c61318 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:16.087247 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd82acd-5180-43d8-8983-0505b7c61318 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:16.087559 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd82acd-5180-43d8-8983-0505b7c61318 None admin] PATH_INFO: `/v3/projects/9d8b38972ba64f509734ed17f30c88fd` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:16.089078 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd82acd-5180-43d8-8983-0505b7c61318 None admin] RBAC: Authorizing `identity:get_project(project_id=9d8b38972ba64f509734ed17f30c88fd)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:16.090930 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd82acd-5180-43d8-8983-0505b7c61318 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:16.093081 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 134/267] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:01:15 2022] GET /identity/v3/projects/9d8b38972ba64f509734ed17f30c88fd => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:16.102948 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd82acd-5180-43d8-8983-0505b7c61318 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:16.103481 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:16.117584 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f952e-f5a3-495b-8ff4-e60a219f38f0 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:16.169470 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:16.212184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7250bac-1def-4862-9846-034fa3a4c512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:16.212842 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7250bac-1def-4862-9846-034fa3a4c512 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:16.213120 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7250bac-1def-4862-9846-034fa3a4c512 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:16.213390 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7250bac-1def-4862-9846-034fa3a4c512 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:16.214118 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7250bac-1def-4862-9846-034fa3a4c512 None admin] RBAC: Adding query filter params (role.id=c53a0e13b58642f291c856efe1f0b87d, scope.project.id=9d8b38972ba64f509734ed17f30c88fd, user.id=15d11995738c4ded976589903d34b89e) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:16.214431 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7250bac-1def-4862-9846-034fa3a4c512 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:16.216459 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7250bac-1def-4862-9846-034fa3a4c512 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:16.226641 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 134/268] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:01:15 2022] GET /identity/v3/role_assignments?user.id=15d11995738c4ded976589903d34b89e&scope.project.id=9d8b38972ba64f509734ed17f30c88fd&role.id=c53a0e13b58642f291c856efe1f0b87d => generated 608 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:01:16.267848 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd82acd-5180-43d8-8983-0505b7c61318 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:16.306933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64bfaf1-b662-4fa6-b0e4-2ca78498477d None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:16.307494 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64bfaf1-b662-4fa6-b0e4-2ca78498477d None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:16.307494 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64bfaf1-b662-4fa6-b0e4-2ca78498477d None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:16.311543 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 135/269] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 12:01:16.321914 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9910ad-fa63-4657-990e-5f7c5ba0b9ee None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:16.322221 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9910ad-fa63-4657-990e-5f7c5ba0b9ee None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:16.322470 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9910ad-fa63-4657-990e-5f7c5ba0b9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:16.340320 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e9a51-a8ce-4199-a6f7-d0b0f11e2983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:16.342183 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e9a51-a8ce-4199-a6f7-d0b0f11e2983 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:16.342183 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e9a51-a8ce-4199-a6f7-d0b0f11e2983 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:16.342183 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e9a51-a8ce-4199-a6f7-d0b0f11e2983 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:16.342796 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5e9a51-a8ce-4199-a6f7-d0b0f11e2983 None admin] RBAC: Adding query filter params (role.id=3a4b7e53b8644e508ac440f71138be16, scope.project.id=9d8b38972ba64f509734ed17f30c88fd, user.id=15d11995738c4ded976589903d34b89e) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:16.343105 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5e9a51-a8ce-4199-a6f7-d0b0f11e2983 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:16.344717 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5e9a51-a8ce-4199-a6f7-d0b0f11e2983 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:16.355191 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 135/270] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:01:16 2022] GET /identity/v3/role_assignments?user.id=15d11995738c4ded976589903d34b89e&scope.project.id=9d8b38972ba64f509734ed17f30c88fd&role.id=3a4b7e53b8644e508ac440f71138be16 => generated 608 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:01:16.360653 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-0f9910ad-fa63-4657-990e-5f7c5ba0b9ee None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:16.361945 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-0f9910ad-fa63-4657-990e-5f7c5ba0b9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:16.479934 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 136/271] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 12:01:16.614678 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765edfdf-4664-4ba7-83cd-d7fbc1989f71 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:16.614895 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765edfdf-4664-4ba7-83cd-d7fbc1989f71 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:16.615181 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765edfdf-4664-4ba7-83cd-d7fbc1989f71 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:16.616849 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 136/272] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:16.621885 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8936aaa1-20ed-4e48-ad31-ceaec982d1da None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:16.622526 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8936aaa1-20ed-4e48-ad31-ceaec982d1da None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:16.622821 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8936aaa1-20ed-4e48-ad31-ceaec982d1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:16.647821 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-8936aaa1-20ed-4e48-ad31-ceaec982d1da None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:16.649036 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-8936aaa1-20ed-4e48-ad31-ceaec982d1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:16.738281 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 137/273] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:16.838730 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765edfdf-4664-4ba7-83cd-d7fbc1989f71 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:16.953811 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8439e62-c21b-4d62-bca7-4957bdd5d1d6 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:16.954174 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8439e62-c21b-4d62-bca7-4957bdd5d1d6 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:16.954290 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8439e62-c21b-4d62-bca7-4957bdd5d1d6 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:16.955730 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 138/274] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 12:01:16.962595 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59aa8d5-4936-4902-a5bc-a866e33fa188 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:16.962882 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59aa8d5-4936-4902-a5bc-a866e33fa188 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:16.963158 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59aa8d5-4936-4902-a5bc-a866e33fa188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:16.964579 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765edfdf-4664-4ba7-83cd-d7fbc1989f71 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:16.966883 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:16.989601 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-d59aa8d5-4936-4902-a5bc-a866e33fa188 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:16.991124 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-d59aa8d5-4936-4902-a5bc-a866e33fa188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:17.013280 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52643f78-7a81-4d18-bc01-04c526f4e9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:17.013925 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52643f78-7a81-4d18-bc01-04c526f4e9ef None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:17.014211 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52643f78-7a81-4d18-bc01-04c526f4e9ef None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:17.014498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52643f78-7a81-4d18-bc01-04c526f4e9ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:17.019720 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:17.031970 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52643f78-7a81-4d18-bc01-04c526f4e9ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:17.039081 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-52643f78-7a81-4d18-bc01-04c526f4e9ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 12:01:17.039081 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:17.039081 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:17.039081 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:17.039081 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:17.039081 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:17.039081 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:17.039081 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:17.039081 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:17.039081 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:17.039081 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:17.039081 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:17.039081 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:17.039081 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:17.039081 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:17.039081 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:17.040012 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:17.040012 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:17.040012 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:17.040012 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:17.040012 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:17.040012 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:17.040012 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:17.040012 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:17.040012 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:17.040012 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:17.040012 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:17.040012 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:17.040012 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:17.040012 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:17.040012 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:17.040012 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:17.040798 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:17.040798 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:17.040798 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:17.040798 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:17.040798 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:17.040798 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:17.040798 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:17.040798 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:17.040798 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:17.040798 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:17.040798 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 12:01:17.040798 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:17.041588 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 137/275] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:01:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:17.042409 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52643f78-7a81-4d18-bc01-04c526f4e9ef None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:17.078224 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 139/276] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:17.079186 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59aa8d5-4936-4902-a5bc-a866e33fa188 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:17.158025 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52643f78-7a81-4d18-bc01-04c526f4e9ef None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:17.184080 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59aa8d5-4936-4902-a5bc-a866e33fa188 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:17.197256 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670fe44-ed0d-492f-afd5-664cc6dc8b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:17.198243 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670fe44-ed0d-492f-afd5-664cc6dc8b22 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:17.198770 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670fe44-ed0d-492f-afd5-664cc6dc8b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:17.199217 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670fe44-ed0d-492f-afd5-664cc6dc8b22 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:17.201075 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a670fe44-ed0d-492f-afd5-664cc6dc8b22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:17.202900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a670fe44-ed0d-492f-afd5-664cc6dc8b22 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:17.204883 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 138/277] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:01:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:17.210439 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670fe44-ed0d-492f-afd5-664cc6dc8b22 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:17.220468 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f1fcd-dbc3-48fd-9e0a-f68078e4f11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:17.221278 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f1fcd-dbc3-48fd-9e0a-f68078e4f11c None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:17.221449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f1fcd-dbc3-48fd-9e0a-f68078e4f11c None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:17.221835 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f1fcd-dbc3-48fd-9e0a-f68078e4f11c None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:17.222563 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f1fcd-dbc3-48fd-9e0a-f68078e4f11c None admin] RBAC: Adding query filter params (name=member) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:17.222936 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f1fcd-dbc3-48fd-9e0a-f68078e4f11c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:17.224448 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f1fcd-dbc3-48fd-9e0a-f68078e4f11c None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:17.237505 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 140/278] 10.222.0.98 () {56 vars in 1101 bytes} [Thu Aug 18 12:01:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:17.238561 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f1fcd-dbc3-48fd-9e0a-f68078e4f11c None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:17.256016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:17.258518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:17.334832 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670fe44-ed0d-492f-afd5-664cc6dc8b22 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:17.349008 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f1fcd-dbc3-48fd-9e0a-f68078e4f11c None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:17.374123 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da834a59-a997-4091-97df-8b62cec9a6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:17.375085 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da834a59-a997-4091-97df-8b62cec9a6f4 None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:17.375600 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da834a59-a997-4091-97df-8b62cec9a6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:17.375877 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da834a59-a997-4091-97df-8b62cec9a6f4 None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:17.376524 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da834a59-a997-4091-97df-8b62cec9a6f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:17.377960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da834a59-a997-4091-97df-8b62cec9a6f4 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:17.384836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2461d285-9234-4372-9834-7d29597a5c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:17.385478 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461d285-9234-4372-9834-7d29597a5c20 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:17.385759 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461d285-9234-4372-9834-7d29597a5c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:17.386029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461d285-9234-4372-9834-7d29597a5c20 None admin] PATH_INFO: `/v3/users/d78d70f2a5cd482eab01c3da40a7b63a` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:17.401685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2461d285-9234-4372-9834-7d29597a5c20 None admin] RBAC: Authorizing `identity:get_user(user_id=d78d70f2a5cd482eab01c3da40a7b63a)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:17.403159 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2461d285-9234-4372-9834-7d29597a5c20 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:17.405406 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 141/279] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:01:17 2022] GET /identity/v3/users/d78d70f2a5cd482eab01c3da40a7b63a => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:17.406202 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2461d285-9234-4372-9834-7d29597a5c20 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:17.477230 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:17.480445 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 139/280] 10.222.0.98 () {60 vars in 1132 bytes} [Thu Aug 18 12:01:17 2022] POST /identity/v3/users => generated 293 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:01:17.481364 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da834a59-a997-4091-97df-8b62cec9a6f4 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:17.512398 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2461d285-9234-4372-9834-7d29597a5c20 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:17.563040 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c59951-5c77-44fc-a95f-d6c182339d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:17.564540 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c59951-5c77-44fc-a95f-d6c182339d08 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:17.565268 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c59951-5c77-44fc-a95f-d6c182339d08 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:17.565701 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c59951-5c77-44fc-a95f-d6c182339d08 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:17.582519 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c59951-5c77-44fc-a95f-d6c182339d08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:17.589809 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-94c59951-5c77-44fc-a95f-d6c182339d08 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 12:01:17.589809 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:17.589809 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:17.589809 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:17.589809 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:17.589809 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:17.589809 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:17.589809 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:17.589809 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:17.589809 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:17.589809 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:17.589809 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:17.589809 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:17.589809 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:17.589809 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:17.589809 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:17.590884 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:17.590884 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:17.590884 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:17.590884 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:17.590884 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:17.590884 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:17.590884 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:17.590884 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:17.590884 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:17.590884 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:17.590884 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:17.590884 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:17.590884 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:17.590884 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:17.590884 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:17.590884 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:17.591726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:17.591726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:17.591726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:17.591726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:17.591726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:17.591726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:17.591726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:17.591726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:17.591726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:17.591726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:17.591726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 12:01:17.591726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:17.592323 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 142/281] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:01:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:17.593705 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1285b2-9f5d-413a-ac9b-6d49a4213414 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:17.594054 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1285b2-9f5d-413a-ac9b-6d49a4213414 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:17.594341 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1285b2-9f5d-413a-ac9b-6d49a4213414 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:17.595708 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 143/282] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:17.600075 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:17.600503 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1285b2-9f5d-413a-ac9b-6d49a4213414 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:17.620450 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da834a59-a997-4091-97df-8b62cec9a6f4 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:17.662038 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ae6d6-62a8-45d4-b1d2-e251a162b35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:17.662659 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ae6d6-62a8-45d4-b1d2-e251a162b35b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:17.662930 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ae6d6-62a8-45d4-b1d2-e251a162b35b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:17.663207 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ae6d6-62a8-45d4-b1d2-e251a162b35b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:17.664011 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790ae6d6-62a8-45d4-b1d2-e251a162b35b None admin] RBAC: Adding query filter params (role.id=c81e28908b9b49a38d33978df4d11716, user.id=d78d70f2a5cd482eab01c3da40a7b63a, scope.system=all) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:17.664324 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790ae6d6-62a8-45d4-b1d2-e251a162b35b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:17.665761 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790ae6d6-62a8-45d4-b1d2-e251a162b35b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:17.672349 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 140/283] 10.222.0.98 () {56 vars in 1308 bytes} [Thu Aug 18 12:01:17 2022] GET /identity/v3/role_assignments?user.id=d78d70f2a5cd482eab01c3da40a7b63a&scope.system=all&role.id=c81e28908b9b49a38d33978df4d11716 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:17.673563 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6bc1e-30e0-4b4e-bd06-e266b9047752 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:17.673851 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6bc1e-30e0-4b4e-bd06-e266b9047752 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:17.674123 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6bc1e-30e0-4b4e-bd06-e266b9047752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:17.705798 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-4ed6bc1e-30e0-4b4e-bd06-e266b9047752 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:17.707463 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-4ed6bc1e-30e0-4b4e-bd06-e266b9047752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:17.746323 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1285b2-9f5d-413a-ac9b-6d49a4213414 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:17.790653 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:17.797787 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f061da38-ba52-4440-bf80-6b0c25a39fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:17.798480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061da38-ba52-4440-bf80-6b0c25a39fc9 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:17.798760 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061da38-ba52-4440-bf80-6b0c25a39fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:17.799032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061da38-ba52-4440-bf80-6b0c25a39fc9 None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:17.799965 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f061da38-ba52-4440-bf80-6b0c25a39fc9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:17.800273 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f061da38-ba52-4440-bf80-6b0c25a39fc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:17.801784 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f061da38-ba52-4440-bf80-6b0c25a39fc9 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:17.812107 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 141/284] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:17.812693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 144/285] 10.222.0.98 () {56 vars in 1101 bytes} [Thu Aug 18 12:01:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:17.820248 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6bc1e-30e0-4b4e-bd06-e266b9047752 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:17.917268 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:17.938986 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6bc1e-30e0-4b4e-bd06-e266b9047752 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:17.979750 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6cf72e-378f-49f0-af13-b9cac22c076d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:17.980401 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6cf72e-378f-49f0-af13-b9cac22c076d None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:17.980796 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6cf72e-378f-49f0-af13-b9cac22c076d None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:17.981078 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6cf72e-378f-49f0-af13-b9cac22c076d None admin] PATH_INFO: `/v3/users/0fc76e658e5046719fd84efc3749bdce` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:17.982856 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6cf72e-378f-49f0-af13-b9cac22c076d None admin] RBAC: Authorizing `identity:get_user(user_id=0fc76e658e5046719fd84efc3749bdce)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:17.984403 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6cf72e-378f-49f0-af13-b9cac22c076d None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:17.986058 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 142/286] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:01:17 2022] GET /identity/v3/users/0fc76e658e5046719fd84efc3749bdce => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:17.991747 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f061da38-ba52-4440-bf80-6b0c25a39fc9 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:18.132671 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6cf72e-378f-49f0-af13-b9cac22c076d None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:18.145122 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f061da38-ba52-4440-bf80-6b0c25a39fc9 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:18.187504 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9efa7e5-07ed-4467-a576-c8d02294aa41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:18.187504 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9efa7e5-07ed-4467-a576-c8d02294aa41 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:18.187504 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9efa7e5-07ed-4467-a576-c8d02294aa41 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:18.187504 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9efa7e5-07ed-4467-a576-c8d02294aa41 None admin] PATH_INFO: `/v3/projects/9d8b38972ba64f509734ed17f30c88fd` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:18.188098 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9efa7e5-07ed-4467-a576-c8d02294aa41 None admin] RBAC: Authorizing `identity:get_project(project_id=9d8b38972ba64f509734ed17f30c88fd)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:18.191824 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9efa7e5-07ed-4467-a576-c8d02294aa41 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:18.193553 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 145/287] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:01:17 2022] GET /identity/v3/projects/9d8b38972ba64f509734ed17f30c88fd => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:18.201310 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9efa7e5-07ed-4467-a576-c8d02294aa41 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:18.301005 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6cf72e-378f-49f0-af13-b9cac22c076d None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:18.320453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:18.363225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7796354b-55e6-4eda-8301-4452bc1494c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:18.363905 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796354b-55e6-4eda-8301-4452bc1494c4 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:18.364199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796354b-55e6-4eda-8301-4452bc1494c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:18.364711 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796354b-55e6-4eda-8301-4452bc1494c4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:18.373886 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9efa7e5-07ed-4467-a576-c8d02294aa41 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:18.392164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7796354b-55e6-4eda-8301-4452bc1494c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:18.397294 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-7796354b-55e6-4eda-8301-4452bc1494c4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 12:01:18.397294 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:18.397294 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:18.397294 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:18.397294 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:18.397294 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:18.397294 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:18.397294 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:18.397294 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:18.397294 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:18.397294 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:18.397294 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:18.397294 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:18.397294 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:18.397294 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:18.397294 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:18.398240 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:18.398240 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:18.398240 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:18.398240 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:18.398240 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:18.398240 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:18.398240 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:18.398240 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:18.398240 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:18.398240 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:18.398240 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:18.398240 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:18.398240 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:18.398240 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:18.398240 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:18.398240 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:18.400137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:18.400137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:18.400137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:18.400137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:18.400137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:18.400137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:18.400137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:18.400137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:18.400137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:18.400137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:18.400137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 12:01:18.400137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:18.400137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 143/288] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:01:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:01:18.416546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7796354b-55e6-4eda-8301-4452bc1494c4 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:18.419594 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943130e-3ab2-4b40-917e-649b0531759d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:18.420379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943130e-3ab2-4b40-917e-649b0531759d None admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:18.420737 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943130e-3ab2-4b40-917e-649b0531759d None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:18.421075 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943130e-3ab2-4b40-917e-649b0531759d None admin] PATH_INFO: `/v3/projects/9d8b38972ba64f509734ed17f30c88fd/users/0fc76e658e5046719fd84efc3749bdce/roles/93cdab4d25f54b8ca825e7ee5a5dc2d2` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:18.423321 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0943130e-3ab2-4b40-917e-649b0531759d None admin] RBAC: Authorizing `identity:create_grant(project_id=9d8b38972ba64f509734ed17f30c88fd, user_id=0fc76e658e5046719fd84efc3749bdce, role_id=93cdab4d25f54b8ca825e7ee5a5dc2d2)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:18.425850 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0943130e-3ab2-4b40-917e-649b0531759d None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:18.435804 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:18.522232 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 146/289] 10.222.0.98 () {58 vars in 1439 bytes} [Thu Aug 18 12:01:18 2022] PUT /identity/v3/projects/9d8b38972ba64f509734ed17f30c88fd/users/0fc76e658e5046719fd84efc3749bdce/roles/93cdab4d25f54b8ca825e7ee5a5dc2d2 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:01:18.540621 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7796354b-55e6-4eda-8301-4452bc1494c4 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:18.581608 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1968f5fe-fbbb-4295-9d7b-ec2cf7378d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:18.582529 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968f5fe-fbbb-4295-9d7b-ec2cf7378d7a None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:18.582986 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968f5fe-fbbb-4295-9d7b-ec2cf7378d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:18.583497 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968f5fe-fbbb-4295-9d7b-ec2cf7378d7a None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:18.584471 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1968f5fe-fbbb-4295-9d7b-ec2cf7378d7a None admin] RBAC: Adding query filter params (name=member) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:18.585024 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1968f5fe-fbbb-4295-9d7b-ec2cf7378d7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:18.586623 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1968f5fe-fbbb-4295-9d7b-ec2cf7378d7a None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:18.597407 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 144/290] 10.222.0.98 () {56 vars in 1101 bytes} [Thu Aug 18 12:01:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:18.603048 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943130e-3ab2-4b40-917e-649b0531759d None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:18.746251 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943130e-3ab2-4b40-917e-649b0531759d None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:18.770819 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:18.788277 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f139e06b-8873-4489-bb9c-c1d278cf6d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:18.789310 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139e06b-8873-4489-bb9c-c1d278cf6d73 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:18.789800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139e06b-8873-4489-bb9c-c1d278cf6d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:18.790303 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139e06b-8873-4489-bb9c-c1d278cf6d73 None admin] PATH_INFO: `/v3/users/9a0ca00de2084bb2818c40132bc31936` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:18.791710 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f139e06b-8873-4489-bb9c-c1d278cf6d73 None admin] RBAC: Authorizing `identity:get_user(user_id=9a0ca00de2084bb2818c40132bc31936)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:18.794643 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f139e06b-8873-4489-bb9c-c1d278cf6d73 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:18.796923 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 147/291] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:01:18 2022] GET /identity/v3/users/9a0ca00de2084bb2818c40132bc31936 => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:18.806145 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1968f5fe-fbbb-4295-9d7b-ec2cf7378d7a None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:18.900079 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:18.939119 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1968f5fe-fbbb-4295-9d7b-ec2cf7378d7a None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:18.992794 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acafd1ae-2560-49a3-af8d-1b35a67b52e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:18.993420 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafd1ae-2560-49a3-af8d-1b35a67b52e6 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:18.993709 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafd1ae-2560-49a3-af8d-1b35a67b52e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:18.993988 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafd1ae-2560-49a3-af8d-1b35a67b52e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:19.074514 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acafd1ae-2560-49a3-af8d-1b35a67b52e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:19.076643 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acafd1ae-2560-49a3-af8d-1b35a67b52e6 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:19.174475 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:19.207429 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 145/292] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:01:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:01:19.213411 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f139e06b-8873-4489-bb9c-c1d278cf6d73 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:19.339286 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f139e06b-8873-4489-bb9c-c1d278cf6d73 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:19.380271 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be551a67-a876-489e-8a90-5580815a17e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:19.380884 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be551a67-a876-489e-8a90-5580815a17e2 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:19.381176 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be551a67-a876-489e-8a90-5580815a17e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:19.381453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be551a67-a876-489e-8a90-5580815a17e2 None admin] PATH_INFO: `/v3/projects/9d8b38972ba64f509734ed17f30c88fd` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:19.382601 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be551a67-a876-489e-8a90-5580815a17e2 None admin] RBAC: Authorizing `identity:get_project(project_id=9d8b38972ba64f509734ed17f30c88fd)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:19.384092 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be551a67-a876-489e-8a90-5580815a17e2 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:19.385898 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 148/293] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:01:19 2022] GET /identity/v3/projects/9d8b38972ba64f509734ed17f30c88fd => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:19.391849 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acafd1ae-2560-49a3-af8d-1b35a67b52e6 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:19.503178 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acafd1ae-2560-49a3-af8d-1b35a67b52e6 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:19.540657 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26414f8a-ca9d-4651-89dd-73efba9b0b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:19.541271 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26414f8a-ca9d-4651-89dd-73efba9b0b70 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:19.541554 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26414f8a-ca9d-4651-89dd-73efba9b0b70 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:19.541830 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26414f8a-ca9d-4651-89dd-73efba9b0b70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:19.542568 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26414f8a-ca9d-4651-89dd-73efba9b0b70 None admin] RBAC: Adding query filter params (role.id=c81e28908b9b49a38d33978df4d11716, scope.project.id=9d8b38972ba64f509734ed17f30c88fd, user.id=9a0ca00de2084bb2818c40132bc31936) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:19.542880 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26414f8a-ca9d-4651-89dd-73efba9b0b70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:19.545283 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26414f8a-ca9d-4651-89dd-73efba9b0b70 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:19.554568 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 146/294] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:01:19 2022] GET /identity/v3/role_assignments?user.id=9a0ca00de2084bb2818c40132bc31936&scope.project.id=9d8b38972ba64f509734ed17f30c88fd&role.id=c81e28908b9b49a38d33978df4d11716 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:19.910708 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d356659-0fdc-4e41-b030-ca5dab34fa29 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:19.911104 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d356659-0fdc-4e41-b030-ca5dab34fa29 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:19.911282 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d356659-0fdc-4e41-b030-ca5dab34fa29 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:19.912886 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 149/295] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:19.919200 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d4e1c-b385-4683-8f6c-40abb9fa72fb None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:19.920032 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d4e1c-b385-4683-8f6c-40abb9fa72fb None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:19.920317 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d4e1c-b385-4683-8f6c-40abb9fa72fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:19.941224 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-a50d4e1c-b385-4683-8f6c-40abb9fa72fb None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:19.942450 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-a50d4e1c-b385-4683-8f6c-40abb9fa72fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:19.974716 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:20.022720 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 147/296] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:20.073240 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ebf8bf-3868-4711-b686-70b0436077fb None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:20.073523 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ebf8bf-3868-4711-b686-70b0436077fb None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:20.073806 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ebf8bf-3868-4711-b686-70b0436077fb None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:20.075130 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 150/297] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:20.080628 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b33d8-0ff2-45e5-a668-cc8b217ea30d None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:20.080927 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b33d8-0ff2-45e5-a668-cc8b217ea30d None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:20.081208 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b33d8-0ff2-45e5-a668-cc8b217ea30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:20.102021 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-bc1b33d8-0ff2-45e5-a668-cc8b217ea30d None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:20.103234 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-bc1b33d8-0ff2-45e5-a668-cc8b217ea30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:20.185494 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 148/298] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:20.223560 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e1213-19e8-4774-b30a-86f2d7941ab5 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:20.223875 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e1213-19e8-4774-b30a-86f2d7941ab5 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:20.224122 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e1213-19e8-4774-b30a-86f2d7941ab5 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:20.225867 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 151/299] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:20.231229 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887ae62-02b3-4473-b2e6-32be4dfebfbe None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:20.232143 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887ae62-02b3-4473-b2e6-32be4dfebfbe None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:20.232440 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887ae62-02b3-4473-b2e6-32be4dfebfbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:20.251268 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-7887ae62-02b3-4473-b2e6-32be4dfebfbe None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:20.252500 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-7887ae62-02b3-4473-b2e6-32be4dfebfbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:20.322036 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e1213-19e8-4774-b30a-86f2d7941ab5 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:20.324903 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 149/300] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:20.384727 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:20.427270 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e1213-19e8-4774-b30a-86f2d7941ab5 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:20.454456 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7887ae62-02b3-4473-b2e6-32be4dfebfbe None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:20.463759 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c9c93e-db54-4021-80f4-34e6ddb2502c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:20.464404 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9c93e-db54-4021-80f4-34e6ddb2502c None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:20.464683 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9c93e-db54-4021-80f4-34e6ddb2502c None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:20.464955 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9c93e-db54-4021-80f4-34e6ddb2502c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:20.476910 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c9c93e-db54-4021-80f4-34e6ddb2502c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:20.480091 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:20.483696 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-60c9c93e-db54-4021-80f4-34e6ddb2502c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 12:01:20.483696 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:20.483696 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:20.483696 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:20.483696 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:20.483696 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:20.483696 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:20.483696 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:20.483696 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:20.483696 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:20.483696 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:20.483696 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:20.483696 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:20.483696 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:20.483696 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:20.483696 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:20.484770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:20.484770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:20.484770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:20.484770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:20.484770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:20.484770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:20.484770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:20.484770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:20.484770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:20.484770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:20.484770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:20.484770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:20.484770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:20.484770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:20.484770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:20.484770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:20.485851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:20.485851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:20.485851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:20.485851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:20.485851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:20.485851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:20.485851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:20.485851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:20.485851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:20.485851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:20.485851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 12:01:20.485851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:20.485851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 152/301] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:01:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:20.492445 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c9c93e-db54-4021-80f4-34e6ddb2502c None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:20.577738 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7887ae62-02b3-4473-b2e6-32be4dfebfbe None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:20.615926 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e82bf2-1864-40fb-8724-3f07f3c220ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:20.616680 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e82bf2-1864-40fb-8724-3f07f3c220ae None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:20.617032 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e82bf2-1864-40fb-8724-3f07f3c220ae None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:20.617410 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e82bf2-1864-40fb-8724-3f07f3c220ae None admin] PATH_INFO: `/v3/domains/default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:20.618258 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c9c93e-db54-4021-80f4-34e6ddb2502c None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:20.618650 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e82bf2-1864-40fb-8724-3f07f3c220ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:20.620199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e82bf2-1864-40fb-8724-3f07f3c220ae None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:20.621922 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 150/302] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:01:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:01:20.622659 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e82bf2-1864-40fb-8724-3f07f3c220ae None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:20.638551 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:20.657013 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc470e22-ae1f-4edc-be00-31868b362397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:20.657622 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc470e22-ae1f-4edc-be00-31868b362397 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:20.657907 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc470e22-ae1f-4edc-be00-31868b362397 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:20.658184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc470e22-ae1f-4edc-be00-31868b362397 None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:20.658850 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc470e22-ae1f-4edc-be00-31868b362397 None admin] RBAC: Adding query filter params (name=member) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:20.659158 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc470e22-ae1f-4edc-be00-31868b362397 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:20.660567 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc470e22-ae1f-4edc-be00-31868b362397 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:20.674570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 153/303] 10.222.0.98 () {56 vars in 1101 bytes} [Thu Aug 18 12:01:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:20.675769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc470e22-ae1f-4edc-be00-31868b362397 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:20.729720 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:20.739858 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e82bf2-1864-40fb-8724-3f07f3c220ae None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:20.779334 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e31aa93-cfdc-4a71-bd41-d677582fe9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:20.780633 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31aa93-cfdc-4a71-bd41-d677582fe9d8 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:20.781509 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31aa93-cfdc-4a71-bd41-d677582fe9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:20.782272 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31aa93-cfdc-4a71-bd41-d677582fe9d8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:20.789738 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc470e22-ae1f-4edc-be00-31868b362397 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:20.794031 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e31aa93-cfdc-4a71-bd41-d677582fe9d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:20.799238 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-2e31aa93-cfdc-4a71-bd41-d677582fe9d8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 12:01:20.799238 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:20.799238 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:20.799238 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:20.799238 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:20.799238 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:20.799238 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:20.799238 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:20.799238 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:20.799238 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:20.799238 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:20.799238 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:20.799238 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:20.799238 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:20.799238 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:20.799238 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:20.800308 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:20.800308 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:20.800308 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:20.800308 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:20.800308 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:20.800308 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:20.800308 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:20.800308 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:20.800308 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:20.800308 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:20.800308 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:20.800308 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:20.800308 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:20.800308 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:20.800308 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:20.800308 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:20.801407 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:20.801407 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:20.801407 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:20.801407 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:20.801407 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:20.801407 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:20.801407 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:20.801407 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:20.801407 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:20.801407 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:20.801407 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 12:01:20.801407 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:20.801407 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 151/304] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:01:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:20.801407 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e31aa93-cfdc-4a71-bd41-d677582fe9d8 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:20.831605 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038e9d26-e34e-47aa-9e6b-22079a564a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:20.832311 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e9d26-e34e-47aa-9e6b-22079a564a48 None admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:20.832593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e9d26-e34e-47aa-9e6b-22079a564a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:20.832896 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e9d26-e34e-47aa-9e6b-22079a564a48 None admin] PATH_INFO: `/v3/groups` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:20.833531 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038e9d26-e34e-47aa-9e6b-22079a564a48 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:20.834970 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038e9d26-e34e-47aa-9e6b-22079a564a48 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:20.914655 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e31aa93-cfdc-4a71-bd41-d677582fe9d8 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:20.921587 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 154/305] 10.222.0.98 () {60 vars in 1134 bytes} [Thu Aug 18 12:01:20 2022] POST /identity/v3/groups => generated 229 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:01:20.922465 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038e9d26-e34e-47aa-9e6b-22079a564a48 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:20.970314 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5c354-9548-412c-b310-f24ea32ed91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:20.971578 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5c354-9548-412c-b310-f24ea32ed91d None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:20.972094 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5c354-9548-412c-b310-f24ea32ed91d None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:20.972584 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5c354-9548-412c-b310-f24ea32ed91d None admin] PATH_INFO: `/v3/users/d78d70f2a5cd482eab01c3da40a7b63a` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:20.974793 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c5c354-9548-412c-b310-f24ea32ed91d None admin] RBAC: Authorizing `identity:get_user(user_id=d78d70f2a5cd482eab01c3da40a7b63a)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:20.977452 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:20.977941 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c5c354-9548-412c-b310-f24ea32ed91d None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:20.986596 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 152/306] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:01:20 2022] GET /identity/v3/users/d78d70f2a5cd482eab01c3da40a7b63a => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:20.990307 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5c354-9548-412c-b310-f24ea32ed91d None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:21.024297 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:21.066297 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038e9d26-e34e-47aa-9e6b-22079a564a48 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:21.110511 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d288be-4725-4ef1-a8e4-42fdcbbafebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:21.111122 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d288be-4725-4ef1-a8e4-42fdcbbafebb None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:21.111402 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d288be-4725-4ef1-a8e4-42fdcbbafebb None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:21.111725 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d288be-4725-4ef1-a8e4-42fdcbbafebb None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:21.112396 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d288be-4725-4ef1-a8e4-42fdcbbafebb None admin] RBAC: Adding query filter params (name=reader) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:21.112702 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d288be-4725-4ef1-a8e4-42fdcbbafebb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:21.114091 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d288be-4725-4ef1-a8e4-42fdcbbafebb None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:21.124336 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 155/307] 10.222.0.98 () {56 vars in 1101 bytes} [Thu Aug 18 12:01:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:21.128711 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1d432-888d-4eeb-ab2e-543d43d0c65b None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:21.129211 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1d432-888d-4eeb-ab2e-543d43d0c65b None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:21.129495 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1d432-888d-4eeb-ab2e-543d43d0c65b None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:21.130769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 156/308] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:21.133328 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5c354-9548-412c-b310-f24ea32ed91d None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:21.135598 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c1d432-888d-4eeb-ab2e-543d43d0c65b None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:21.172179 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3f11a-3b74-4b35-841a-820d21fe5f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:21.172843 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3f11a-3b74-4b35-841a-820d21fe5f6c None admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:21.173129 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3f11a-3b74-4b35-841a-820d21fe5f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:21.173415 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3f11a-3b74-4b35-841a-820d21fe5f6c None admin] PATH_INFO: `/v3/system/users/d78d70f2a5cd482eab01c3da40a7b63a/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:21.174861 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d3f11a-3b74-4b35-841a-820d21fe5f6c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d78d70f2a5cd482eab01c3da40a7b63a, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:21.176271 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d3f11a-3b74-4b35-841a-820d21fe5f6c None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:21.266278 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c1d432-888d-4eeb-ab2e-543d43d0c65b None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:21.275635 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 153/309] 10.222.0.98 () {58 vars in 1334 bytes} [Thu Aug 18 12:01:20 2022] PUT /identity/v3/system/users/d78d70f2a5cd482eab01c3da40a7b63a/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:01:21.275840 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:21.275925 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ec9a9-a5f0-463a-9878-6d03e4387643 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:21.275925 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ec9a9-a5f0-463a-9878-6d03e4387643 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:21.275925 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ec9a9-a5f0-463a-9878-6d03e4387643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:21.290433 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-346ec9a9-a5f0-463a-9878-6d03e4387643 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:21.291860 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-346ec9a9-a5f0-463a-9878-6d03e4387643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:21.326475 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe98f3-989f-4987-afa3-03ff57db0015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:21.327426 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe98f3-989f-4987-afa3-03ff57db0015 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:21.327713 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe98f3-989f-4987-afa3-03ff57db0015 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:21.327989 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe98f3-989f-4987-afa3-03ff57db0015 None admin] PATH_INFO: `/v3/users/2e1c4363d82f4988971d67da06a6137a` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:21.352028 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe98f3-989f-4987-afa3-03ff57db0015 None admin] RBAC: Authorizing `identity:get_user(user_id=2e1c4363d82f4988971d67da06a6137a)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:21.353639 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe98f3-989f-4987-afa3-03ff57db0015 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:21.359695 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 157/310] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:01:21 2022] GET /identity/v3/users/2e1c4363d82f4988971d67da06a6137a => generated 293 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:21.363134 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe98f3-989f-4987-afa3-03ff57db0015 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:21.385126 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 154/311] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:21.489842 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe98f3-989f-4987-afa3-03ff57db0015 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:21.528405 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad3aa2-7906-45d8-a0c1-5ebb3a6c89ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:21.528974 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad3aa2-7906-45d8-a0c1-5ebb3a6c89ed None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:21.529257 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad3aa2-7906-45d8-a0c1-5ebb3a6c89ed None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:21.529535 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad3aa2-7906-45d8-a0c1-5ebb3a6c89ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:21.530409 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ad3aa2-7906-45d8-a0c1-5ebb3a6c89ed None admin] RBAC: Adding query filter params (role.id=93cdab4d25f54b8ca825e7ee5a5dc2d2, user.id=2e1c4363d82f4988971d67da06a6137a, scope.system=all) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:21.530727 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ad3aa2-7906-45d8-a0c1-5ebb3a6c89ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:21.532224 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ad3aa2-7906-45d8-a0c1-5ebb3a6c89ed None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:21.537952 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 158/312] 10.222.0.98 () {56 vars in 1308 bytes} [Thu Aug 18 12:01:21 2022] GET /identity/v3/role_assignments?user.id=2e1c4363d82f4988971d67da06a6137a&scope.system=all&role.id=93cdab4d25f54b8ca825e7ee5a5dc2d2 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:01:21.659719 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ec9a9-a5f0-463a-9878-6d03e4387643 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:21.676435 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:21.772964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ec9a9-a5f0-463a-9878-6d03e4387643 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:21.813967 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d494bcb-6881-40cd-bdb4-4d74720c800c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:21.814530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d494bcb-6881-40cd-bdb4-4d74720c800c None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:21.814814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d494bcb-6881-40cd-bdb4-4d74720c800c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:21.815089 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d494bcb-6881-40cd-bdb4-4d74720c800c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:21.825881 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d494bcb-6881-40cd-bdb4-4d74720c800c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:21.830676 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-3d494bcb-6881-40cd-bdb4-4d74720c800c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 12:01:21.830676 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:21.830676 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:21.830676 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:21.830676 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:21.830676 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:21.830676 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:21.830676 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:21.830676 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:21.830676 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:21.830676 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:21.830676 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:21.830676 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:21.830676 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:21.830676 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:21.830676 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:21.831567 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:21.831567 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:21.831567 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:21.831567 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:21.831567 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:21.831567 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:21.831567 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:21.831567 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:21.831567 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:21.831567 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:21.831567 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:21.831567 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:21.831567 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:21.831567 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:21.831567 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:21.831567 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:21.832347 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:21.832347 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:21.832347 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:21.832347 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:21.832347 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:21.832347 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:21.832347 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:21.832347 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:21.832347 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:21.832347 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:21.832347 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 12:01:21.832347 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:21.832347 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 155/313] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:01:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:21.840544 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad3aa2-7906-45d8-a0c1-5ebb3a6c89ed None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:21.868210 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:21.949046 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad3aa2-7906-45d8-a0c1-5ebb3a6c89ed None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:21.986846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9025ea65-983e-495f-b12a-19494da33201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:21.987856 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025ea65-983e-495f-b12a-19494da33201 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:21.988294 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025ea65-983e-495f-b12a-19494da33201 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:21.988722 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025ea65-983e-495f-b12a-19494da33201 None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:21.989535 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9025ea65-983e-495f-b12a-19494da33201 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:21.989986 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9025ea65-983e-495f-b12a-19494da33201 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:21.991541 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9025ea65-983e-495f-b12a-19494da33201 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:22.000956 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 159/314] 10.222.0.98 () {56 vars in 1101 bytes} [Thu Aug 18 12:01:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:22.011449 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d494bcb-6881-40cd-bdb4-4d74720c800c None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:22.091298 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:22.116872 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d494bcb-6881-40cd-bdb4-4d74720c800c None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:22.154616 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a434130-ac7b-473a-b8bb-c7373e8899ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:22.155769 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a434130-ac7b-473a-b8bb-c7373e8899ff None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:22.156059 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a434130-ac7b-473a-b8bb-c7373e8899ff None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:22.156331 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a434130-ac7b-473a-b8bb-c7373e8899ff None admin] PATH_INFO: `/v3/users/0fc76e658e5046719fd84efc3749bdce` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:22.157458 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a434130-ac7b-473a-b8bb-c7373e8899ff None admin] RBAC: Authorizing `identity:get_user(user_id=0fc76e658e5046719fd84efc3749bdce)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:22.158904 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a434130-ac7b-473a-b8bb-c7373e8899ff None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:22.160482 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 156/315] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:01:22 2022] GET /identity/v3/users/0fc76e658e5046719fd84efc3749bdce => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:22.166181 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9025ea65-983e-495f-b12a-19494da33201 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:22.255997 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:22.272207 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9025ea65-983e-495f-b12a-19494da33201 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:22.306835 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8995ff6f-f1b4-462d-ad37-d7fdeea9595d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:22.307467 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995ff6f-f1b4-462d-ad37-d7fdeea9595d None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:22.307752 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995ff6f-f1b4-462d-ad37-d7fdeea9595d None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:22.308024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995ff6f-f1b4-462d-ad37-d7fdeea9595d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:22.379835 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8995ff6f-f1b4-462d-ad37-d7fdeea9595d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:22.381634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8995ff6f-f1b4-462d-ad37-d7fdeea9595d None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:22.471180 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:22.488604 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 160/316] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:01:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:01:22.493890 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a434130-ac7b-473a-b8bb-c7373e8899ff None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:22.626844 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a434130-ac7b-473a-b8bb-c7373e8899ff None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:22.662560 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cbbe64-f0e7-4bbc-a06c-afb047a493a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:22.663185 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbbe64-f0e7-4bbc-a06c-afb047a493a5 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:22.663434 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbbe64-f0e7-4bbc-a06c-afb047a493a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:22.663703 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbbe64-f0e7-4bbc-a06c-afb047a493a5 None admin] PATH_INFO: `/v3/projects/9d8b38972ba64f509734ed17f30c88fd` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:22.664894 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cbbe64-f0e7-4bbc-a06c-afb047a493a5 None admin] RBAC: Authorizing `identity:get_project(project_id=9d8b38972ba64f509734ed17f30c88fd)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:22.666385 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cbbe64-f0e7-4bbc-a06c-afb047a493a5 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:22.668220 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 157/317] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:01:22 2022] GET /identity/v3/projects/9d8b38972ba64f509734ed17f30c88fd => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:22.675054 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8995ff6f-f1b4-462d-ad37-d7fdeea9595d None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:22.791213 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8995ff6f-f1b4-462d-ad37-d7fdeea9595d None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:22.829044 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2cdb2c-56b9-4dbe-a16a-1c33715bebf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:22.829654 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2cdb2c-56b9-4dbe-a16a-1c33715bebf9 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:22.829931 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2cdb2c-56b9-4dbe-a16a-1c33715bebf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:22.830201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2cdb2c-56b9-4dbe-a16a-1c33715bebf9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:22.830920 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2cdb2c-56b9-4dbe-a16a-1c33715bebf9 None admin] RBAC: Adding query filter params (role.id=93cdab4d25f54b8ca825e7ee5a5dc2d2, scope.project.id=9d8b38972ba64f509734ed17f30c88fd, user.id=0fc76e658e5046719fd84efc3749bdce) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:22.831226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2cdb2c-56b9-4dbe-a16a-1c33715bebf9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:22.832682 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2cdb2c-56b9-4dbe-a16a-1c33715bebf9 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:22.839090 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 161/318] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:01:22 2022] GET /identity/v3/role_assignments?user.id=0fc76e658e5046719fd84efc3749bdce&scope.project.id=9d8b38972ba64f509734ed17f30c88fd&role.id=93cdab4d25f54b8ca825e7ee5a5dc2d2 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:22.920164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483c8e5-1319-4265-86b8-2e9d44400fdd None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:22.920406 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483c8e5-1319-4265-86b8-2e9d44400fdd None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:22.920721 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483c8e5-1319-4265-86b8-2e9d44400fdd None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:22.922391 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 158/319] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:22.928402 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a40e3c-859c-49b5-b017-f2f838a1e737 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:22.931918 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a40e3c-859c-49b5-b017-f2f838a1e737 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:22.931918 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a40e3c-859c-49b5-b017-f2f838a1e737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:22.950898 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-a6a40e3c-859c-49b5-b017-f2f838a1e737 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:22.952169 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-a6a40e3c-859c-49b5-b017-f2f838a1e737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:23.006027 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:23.034085 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 162/320] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:23.303335 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483c8e5-1319-4265-86b8-2e9d44400fdd None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:23.316610 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:23.318795 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80d5e9-7e7d-4508-bf9d-908069c88748 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:23.319090 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80d5e9-7e7d-4508-bf9d-908069c88748 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:23.319370 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80d5e9-7e7d-4508-bf9d-908069c88748 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:23.320914 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 163/321] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:23.325754 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483eaa2-8be5-4f07-89f1-8e67318e94c9 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:23.326046 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483eaa2-8be5-4f07-89f1-8e67318e94c9 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:23.326324 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483eaa2-8be5-4f07-89f1-8e67318e94c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:23.346342 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-a483eaa2-8be5-4f07-89f1-8e67318e94c9 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:23.347583 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-a483eaa2-8be5-4f07-89f1-8e67318e94c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:23.412938 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483c8e5-1319-4265-86b8-2e9d44400fdd None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:23.426855 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 164/322] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 12:01:23.454775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd861034-5c55-4123-81a4-b7827fc7ebea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:23.455198 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd861034-5c55-4123-81a4-b7827fc7ebea None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:23.455579 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd861034-5c55-4123-81a4-b7827fc7ebea None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:23.455930 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd861034-5c55-4123-81a4-b7827fc7ebea None admin] PATH_INFO: `/v3/domains/default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:23.457160 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd861034-5c55-4123-81a4-b7827fc7ebea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:23.458663 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd861034-5c55-4123-81a4-b7827fc7ebea None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:23.460316 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 159/323] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:01:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:23.465811 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a483eaa2-8be5-4f07-89f1-8e67318e94c9 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:23.528337 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be93b29-44b3-49e9-851e-e9cc39818969 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:23.528619 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be93b29-44b3-49e9-851e-e9cc39818969 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:23.529099 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be93b29-44b3-49e9-851e-e9cc39818969 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:23.530423 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 160/324] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 12:01:23.535256 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3390294-739f-46cf-8047-1343ba7b909a None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:23.535725 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3390294-739f-46cf-8047-1343ba7b909a None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:23.536002 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3390294-739f-46cf-8047-1343ba7b909a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:23.554364 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-c3390294-739f-46cf-8047-1343ba7b909a None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:23.555480 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-c3390294-739f-46cf-8047-1343ba7b909a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:23.576168 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a483eaa2-8be5-4f07-89f1-8e67318e94c9 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:23.594308 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:23.623205 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd65f12-a7dd-4a06-81b6-48777cf50dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:23.623842 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd65f12-a7dd-4a06-81b6-48777cf50dda None admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:23.624124 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd65f12-a7dd-4a06-81b6-48777cf50dda None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:23.624399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd65f12-a7dd-4a06-81b6-48777cf50dda None admin] PATH_INFO: `/v3/groups` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:23.625022 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd65f12-a7dd-4a06-81b6-48777cf50dda None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:23.626562 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd65f12-a7dd-4a06-81b6-48777cf50dda None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:23.638187 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 161/325] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 12:01:23.708469 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3390294-739f-46cf-8047-1343ba7b909a None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:23.719969 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:23.743450 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 165/326] 10.222.0.98 () {60 vars in 1134 bytes} [Thu Aug 18 12:01:23 2022] POST /identity/v3/groups => generated 226 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:01:23.820548 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3390294-739f-46cf-8047-1343ba7b909a None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:23.859033 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bae068-94e7-483d-9b3e-ea3e326cc194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:23.859711 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bae068-94e7-483d-9b3e-ea3e326cc194 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:23.860050 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bae068-94e7-483d-9b3e-ea3e326cc194 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:23.860260 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bae068-94e7-483d-9b3e-ea3e326cc194 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:23.872498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bae068-94e7-483d-9b3e-ea3e326cc194 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:23.877309 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-69bae068-94e7-483d-9b3e-ea3e326cc194 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 12:01:23.877309 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:23.877309 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:23.877309 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:23.877309 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:23.877309 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:23.877309 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:23.877309 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:23.877309 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:23.877309 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:23.877309 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:23.877309 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:23.877309 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:23.877309 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:23.877309 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:23.877309 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:23.878254 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:23.878254 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:23.878254 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:23.878254 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:23.878254 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:23.878254 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:23.878254 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:23.878254 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:23.878254 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:23.878254 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:23.878254 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:23.878254 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:23.878254 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:23.878254 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:23.878254 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:23.878254 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:23.879085 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:23.879085 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:23.879085 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:23.879085 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:23.879085 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:23.879085 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:23.879085 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:23.879085 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:23.879085 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:23.879085 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:23.879085 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 12:01:23.879085 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:23.879085 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 162/327] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:01:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:01:23.885358 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd65f12-a7dd-4a06-81b6-48777cf50dda None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:23.910184 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bae068-94e7-483d-9b3e-ea3e326cc194 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:23.999317 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:24.011894 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd65f12-a7dd-4a06-81b6-48777cf50dda None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:24.048094 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bae068-94e7-483d-9b3e-ea3e326cc194 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:24.058694 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597ef33-2fc0-4afa-bc82-88de23271577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:24.059318 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597ef33-2fc0-4afa-bc82-88de23271577 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:24.059636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597ef33-2fc0-4afa-bc82-88de23271577 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:24.059916 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597ef33-2fc0-4afa-bc82-88de23271577 None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:24.060629 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3597ef33-2fc0-4afa-bc82-88de23271577 None admin] RBAC: Adding query filter params (name=member) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:24.060942 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3597ef33-2fc0-4afa-bc82-88de23271577 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:24.062338 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3597ef33-2fc0-4afa-bc82-88de23271577 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:24.072435 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 166/328] 10.222.0.98 () {56 vars in 1101 bytes} [Thu Aug 18 12:01:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:24.078266 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597ef33-2fc0-4afa-bc82-88de23271577 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:24.093863 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380b1c3-c523-4ee3-854b-a6f25dce162b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:24.094508 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380b1c3-c523-4ee3-854b-a6f25dce162b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:24.094783 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380b1c3-c523-4ee3-854b-a6f25dce162b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:24.095049 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380b1c3-c523-4ee3-854b-a6f25dce162b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:24.107434 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380b1c3-c523-4ee3-854b-a6f25dce162b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:24.112344 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-4380b1c3-c523-4ee3-854b-a6f25dce162b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 12:01:24.112344 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:24.112344 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:24.112344 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:24.112344 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:24.112344 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:24.112344 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:24.112344 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:24.112344 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:24.112344 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:24.112344 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:24.112344 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:24.112344 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:24.112344 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:24.112344 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:24.112344 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:24.113342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:24.113342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:24.113342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:24.113342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:24.113342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:24.113342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:24.113342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:24.113342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:24.113342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:24.113342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:24.113342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:24.113342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:24.113342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:24.113342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:24.113342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:24.113342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:24.114144 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:24.114144 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:24.114144 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:24.114144 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:24.114144 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:24.114144 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:24.114144 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:24.114144 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:24.114144 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:24.114144 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:24.114144 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 12:01:24.114144 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:24.114144 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 163/329] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:01:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:01:24.119554 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380b1c3-c523-4ee3-854b-a6f25dce162b None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:24.120323 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:24.215608 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597ef33-2fc0-4afa-bc82-88de23271577 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:24.243259 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380b1c3-c523-4ee3-854b-a6f25dce162b None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:24.259169 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde42d6-12bb-4214-9979-faef0d5a2099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:24.259869 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde42d6-12bb-4214-9979-faef0d5a2099 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:24.260158 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde42d6-12bb-4214-9979-faef0d5a2099 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:24.260443 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde42d6-12bb-4214-9979-faef0d5a2099 None admin] PATH_INFO: `/v3/users/d78d70f2a5cd482eab01c3da40a7b63a` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:24.261657 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde42d6-12bb-4214-9979-faef0d5a2099 None admin] RBAC: Authorizing `identity:get_user(user_id=d78d70f2a5cd482eab01c3da40a7b63a)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:24.263851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde42d6-12bb-4214-9979-faef0d5a2099 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:24.265794 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 167/330] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:01:24 2022] GET /identity/v3/users/d78d70f2a5cd482eab01c3da40a7b63a => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:24.275883 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:24.277029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde42d6-12bb-4214-9979-faef0d5a2099 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:24.285344 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785040b-7036-4515-8830-937691e23fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:24.286336 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785040b-7036-4515-8830-937691e23fc9 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:24.286336 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785040b-7036-4515-8830-937691e23fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:24.286503 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785040b-7036-4515-8830-937691e23fc9 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:24.287146 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4785040b-7036-4515-8830-937691e23fc9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:24.287534 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4785040b-7036-4515-8830-937691e23fc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:24.288853 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4785040b-7036-4515-8830-937691e23fc9 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:24.301162 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 164/331] 10.222.0.98 () {56 vars in 1101 bytes} [Thu Aug 18 12:01:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:24.313993 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785040b-7036-4515-8830-937691e23fc9 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:24.420520 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:24.424235 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde42d6-12bb-4214-9979-faef0d5a2099 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:24.447453 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785040b-7036-4515-8830-937691e23fc9 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:24.482434 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88726587-852b-4b80-9b3f-7a4d8de5b234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:24.483211 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88726587-852b-4b80-9b3f-7a4d8de5b234 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:24.483558 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88726587-852b-4b80-9b3f-7a4d8de5b234 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:24.483781 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88726587-852b-4b80-9b3f-7a4d8de5b234 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:24.484513 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88726587-852b-4b80-9b3f-7a4d8de5b234 None admin] RBAC: Adding query filter params (role.id=c81e28908b9b49a38d33978df4d11716, user.id=d78d70f2a5cd482eab01c3da40a7b63a, scope.system=all) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:24.484819 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88726587-852b-4b80-9b3f-7a4d8de5b234 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:24.486261 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88726587-852b-4b80-9b3f-7a4d8de5b234 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:24.493856 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a95f9f-99e0-42ed-9506-e10b9e3aae36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:24.494780 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a95f9f-99e0-42ed-9506-e10b9e3aae36 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:24.495402 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a95f9f-99e0-42ed-9506-e10b9e3aae36 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:24.495863 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a95f9f-99e0-42ed-9506-e10b9e3aae36 None admin] PATH_INFO: `/v3/users/2e1c4363d82f4988971d67da06a6137a` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:24.497149 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a95f9f-99e0-42ed-9506-e10b9e3aae36 None admin] RBAC: Authorizing `identity:get_user(user_id=2e1c4363d82f4988971d67da06a6137a)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:24.499221 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a95f9f-99e0-42ed-9506-e10b9e3aae36 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:24.500263 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 168/332] 10.222.0.98 () {56 vars in 1308 bytes} [Thu Aug 18 12:01:24 2022] GET /identity/v3/role_assignments?user.id=d78d70f2a5cd482eab01c3da40a7b63a&scope.system=all&role.id=c81e28908b9b49a38d33978df4d11716 => generated 510 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:01:24.501766 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 165/333] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:01:24 2022] GET /identity/v3/users/2e1c4363d82f4988971d67da06a6137a => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:24.507457 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88726587-852b-4b80-9b3f-7a4d8de5b234 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:24.632894 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88726587-852b-4b80-9b3f-7a4d8de5b234 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:24.681403 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c41a7-8406-4a20-93c4-e4a7e3ffd95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:24.682366 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c41a7-8406-4a20-93c4-e4a7e3ffd95f None admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:24.682834 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c41a7-8406-4a20-93c4-e4a7e3ffd95f None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:24.683292 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c41a7-8406-4a20-93c4-e4a7e3ffd95f None admin] PATH_INFO: `/v3/system/users/2e1c4363d82f4988971d67da06a6137a/roles/93cdab4d25f54b8ca825e7ee5a5dc2d2` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:24.685040 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6c41a7-8406-4a20-93c4-e4a7e3ffd95f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2e1c4363d82f4988971d67da06a6137a, role_id=93cdab4d25f54b8ca825e7ee5a5dc2d2)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:24.687189 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6c41a7-8406-4a20-93c4-e4a7e3ffd95f None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:24.767790 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 169/334] 10.222.0.98 () {58 vars in 1334 bytes} [Thu Aug 18 12:01:24 2022] PUT /identity/v3/system/users/2e1c4363d82f4988971d67da06a6137a/roles/93cdab4d25f54b8ca825e7ee5a5dc2d2 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:01:26.536964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ce6d8-7e09-4f56-9dcf-22bf837ff48f None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:26.537479 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ce6d8-7e09-4f56-9dcf-22bf837ff48f None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:26.537552 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ce6d8-7e09-4f56-9dcf-22bf837ff48f None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:26.538873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 166/335] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:26.544545 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519303f1-c85f-4b7c-808e-e492966cd1d1 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:26.545538 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519303f1-c85f-4b7c-808e-e492966cd1d1 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:26.545643 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519303f1-c85f-4b7c-808e-e492966cd1d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:26.551320 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39373f-95fe-46ee-9040-91f84704db1e None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:26.551635 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39373f-95fe-46ee-9040-91f84704db1e None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:26.551908 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39373f-95fe-46ee-9040-91f84704db1e None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:26.553482 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 167/336] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:26.558855 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52090064-7890-43d4-8029-1af1ca24b251 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:26.559140 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52090064-7890-43d4-8029-1af1ca24b251 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:26.559458 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52090064-7890-43d4-8029-1af1ca24b251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:26.572591 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-519303f1-c85f-4b7c-808e-e492966cd1d1 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:26.573983 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-519303f1-c85f-4b7c-808e-e492966cd1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:26.592982 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-52090064-7890-43d4-8029-1af1ca24b251 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:26.594182 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-52090064-7890-43d4-8029-1af1ca24b251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:26.597085 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:26.666287 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 170/337] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:26.706593 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:26.724805 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 168/338] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 12:01:27.178543 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda10a45-88a1-4a07-bb43-5f3e266f3265 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:27.179270 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda10a45-88a1-4a07-bb43-5f3e266f3265 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:27.179823 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda10a45-88a1-4a07-bb43-5f3e266f3265 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:27.181696 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 171/339] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:27.187072 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a528a3-57c6-4f61-a0cc-573b61c8e8de None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:27.187363 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a528a3-57c6-4f61-a0cc-573b61c8e8de None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:27.187671 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a528a3-57c6-4f61-a0cc-573b61c8e8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:27.212627 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-97a528a3-57c6-4f61-a0cc-573b61c8e8de None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:27.213870 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-97a528a3-57c6-4f61-a0cc-573b61c8e8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:27.235695 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda10a45-88a1-4a07-bb43-5f3e266f3265 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:27.323053 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 169/340] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:27.357595 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a528a3-57c6-4f61-a0cc-573b61c8e8de None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:27.401224 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda10a45-88a1-4a07-bb43-5f3e266f3265 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:27.458391 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3c5b52-178e-4bbf-9bce-651658fca4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:27.459241 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c5b52-178e-4bbf-9bce-651658fca4f5 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:27.459648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c5b52-178e-4bbf-9bce-651658fca4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:27.459930 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c5b52-178e-4bbf-9bce-651658fca4f5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:27.482501 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3c5b52-178e-4bbf-9bce-651658fca4f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:27.487280 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-ba3c5b52-178e-4bbf-9bce-651658fca4f5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 12:01:27.487280 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:27.487280 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:27.487280 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:27.487280 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:27.487280 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:27.487280 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:27.487280 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:27.487280 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:27.487280 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:27.487280 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:27.487280 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:27.487280 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:27.487280 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:27.487280 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:27.487280 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:27.488201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:27.488201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:27.488201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:27.488201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:27.488201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:27.488201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:27.488201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:27.488201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:27.488201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:27.488201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:27.488201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:27.488201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:27.488201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:27.488201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:27.488201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:27.488201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:27.488988 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:27.488988 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:27.488988 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:27.488988 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:27.488988 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:27.488988 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:27.488988 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:27.488988 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:27.488988 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:27.488988 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:27.488988 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 12:01:27.488988 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:27.488988 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 172/341] 10.222.0.98 () {56 vars in 1114 bytes} [Thu Aug 18 12:01:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:27.490412 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52004ef-abe2-4606-9c62-d98b266dfb25 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:27.490412 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52004ef-abe2-4606-9c62-d98b266dfb25 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:27.490757 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52004ef-abe2-4606-9c62-d98b266dfb25 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:27.493562 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a528a3-57c6-4f61-a0cc-573b61c8e8de None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:27.504064 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 173/342] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:27 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:27.504735 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52004ef-abe2-4606-9c62-d98b266dfb25 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:27.519311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:27.542294 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdcf4a6-dec4-4af3-bc1b-f0d1e3f48818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:27.543146 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdcf4a6-dec4-4af3-bc1b-f0d1e3f48818 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:27.543588 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdcf4a6-dec4-4af3-bc1b-f0d1e3f48818 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:27.543770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:27.544087 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdcf4a6-dec4-4af3-bc1b-f0d1e3f48818 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:27.555913 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdcf4a6-dec4-4af3-bc1b-f0d1e3f48818 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:27.562067 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-dcdcf4a6-dec4-4af3-bc1b-f0d1e3f48818 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 12:01:27.562067 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:27.562067 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:27.562067 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:27.562067 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:27.562067 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:27.562067 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:27.562067 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:27.562067 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:27.562067 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:27.562067 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:27.562067 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:27.562067 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:27.562067 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:27.562067 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:27.562067 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:27.563962 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:27.563962 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:27.563962 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:27.563962 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:27.563962 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:27.563962 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:27.563962 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:27.563962 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:27.563962 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:27.563962 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:27.563962 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:27.563962 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:27.563962 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:27.563962 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:27.563962 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:27.563962 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:27.566009 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:27.566009 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:27.566009 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:27.566009 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:27.566009 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:27.566009 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:27.566009 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:27.566009 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:27.566009 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:27.566009 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:27.566009 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 12:01:27.566009 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:27.566009 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 170/343] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:01:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:01:27.567633 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843d367-be2b-4938-99a9-e12ee6add992 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:27.567633 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843d367-be2b-4938-99a9-e12ee6add992 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:27.567633 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843d367-be2b-4938-99a9-e12ee6add992 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:27.571388 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 171/344] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:27.571388 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30537a2a-7755-437e-8a78-ec075bc0a604 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:27.571388 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30537a2a-7755-437e-8a78-ec075bc0a604 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:27.571388 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30537a2a-7755-437e-8a78-ec075bc0a604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:27.598563 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-30537a2a-7755-437e-8a78-ec075bc0a604 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:27.599872 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-30537a2a-7755-437e-8a78-ec075bc0a604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:27.663878 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52004ef-abe2-4606-9c62-d98b266dfb25 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:27.706907 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 172/345] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:27.709264 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf37623-6799-4f36-9caa-4e04bb884dca None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:27.709851 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf37623-6799-4f36-9caa-4e04bb884dca None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:27.710396 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf37623-6799-4f36-9caa-4e04bb884dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:27.717329 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee29326-84ca-4a83-b025-9afb2b2e608b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:27.718150 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee29326-84ca-4a83-b025-9afb2b2e608b None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:27.719457 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee29326-84ca-4a83-b025-9afb2b2e608b None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:27.719891 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee29326-84ca-4a83-b025-9afb2b2e608b None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:27.720738 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee29326-84ca-4a83-b025-9afb2b2e608b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:27.721195 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee29326-84ca-4a83-b025-9afb2b2e608b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:27.722731 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee29326-84ca-4a83-b025-9afb2b2e608b None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:27.731748 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-7cf37623-6799-4f36-9caa-4e04bb884dca None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:27.733881 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 174/346] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:01:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:27.734009 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-7cf37623-6799-4f36-9caa-4e04bb884dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:27.735015 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee29326-84ca-4a83-b025-9afb2b2e608b None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:27.808303 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 173/347] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:27.809491 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf37623-6799-4f36-9caa-4e04bb884dca None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:27.839964 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:27.867152 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:27.870521 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee29326-84ca-4a83-b025-9afb2b2e608b None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:27.915987 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44531c0-73a1-4476-92dd-de6299248182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:27.916619 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44531c0-73a1-4476-92dd-de6299248182 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:27.916905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44531c0-73a1-4476-92dd-de6299248182 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:27.917182 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44531c0-73a1-4476-92dd-de6299248182 None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:27.917873 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44531c0-73a1-4476-92dd-de6299248182 None admin] RBAC: Adding query filter params (name=member) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:27.918184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44531c0-73a1-4476-92dd-de6299248182 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:27.919617 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44531c0-73a1-4476-92dd-de6299248182 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:27.940458 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 175/348] 10.222.0.98 () {56 vars in 1101 bytes} [Thu Aug 18 12:01:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:27.941339 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44531c0-73a1-4476-92dd-de6299248182 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:27.974719 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf37623-6799-4f36-9caa-4e04bb884dca None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:28.039680 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98781d-d14e-4fb9-b521-f49aa182b83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:28.040309 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98781d-d14e-4fb9-b521-f49aa182b83c None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:28.040594 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98781d-d14e-4fb9-b521-f49aa182b83c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:28.040872 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98781d-d14e-4fb9-b521-f49aa182b83c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:28.103957 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44531c0-73a1-4476-92dd-de6299248182 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:28.139925 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa98781d-d14e-4fb9-b521-f49aa182b83c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:28.142041 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa98781d-d14e-4fb9-b521-f49aa182b83c None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:28.148370 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641591be-b4da-4b6f-a974-73311283b44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:28.149009 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641591be-b4da-4b6f-a974-73311283b44d None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:28.149293 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641591be-b4da-4b6f-a974-73311283b44d None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:28.149569 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641591be-b4da-4b6f-a974-73311283b44d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:28.154197 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:28.166973 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641591be-b4da-4b6f-a974-73311283b44d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:28.172332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-641591be-b4da-4b6f-a974-73311283b44d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 12:01:28.172332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:28.172332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:28.172332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:28.172332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:28.172332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:28.172332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:28.172332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:28.172332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:28.172332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:28.172332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:28.172332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:28.172332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:28.172332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:28.172332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:28.172332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:28.176208 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:28.176208 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:28.176208 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:28.176208 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:28.176208 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:28.176208 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:28.176208 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:28.176208 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:28.176208 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:28.176208 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:28.176208 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:28.176208 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:28.176208 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:28.176208 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:28.176208 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:28.176208 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:28.177016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:28.177016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:28.177016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:28.177016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:28.177016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:28.177016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:28.177016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:28.177016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:28.177016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:28.177016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:28.177016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 12:01:28.177016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:28.177016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 176/349] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:01:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:28.177016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641591be-b4da-4b6f-a974-73311283b44d None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:28.204318 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:28.293174 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 174/350] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:01:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:01:28.293798 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98781d-d14e-4fb9-b521-f49aa182b83c None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:28.313335 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641591be-b4da-4b6f-a974-73311283b44d None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:28.361924 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b50402-d6b3-4dea-aa43-4541054f6dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:28.362612 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b50402-d6b3-4dea-aa43-4541054f6dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:28.362901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b50402-d6b3-4dea-aa43-4541054f6dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:28.363187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b50402-d6b3-4dea-aa43-4541054f6dc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:28.436283 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98781d-d14e-4fb9-b521-f49aa182b83c None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:28.442160 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:28.449051 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b50402-d6b3-4dea-aa43-4541054f6dc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:28.456111 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b50402-d6b3-4dea-aa43-4541054f6dc5 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:28.476494 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961fe83f-e974-47aa-a968-f9b0a5ca321a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:28.477129 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fe83f-e974-47aa-a968-f9b0a5ca321a None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:28.477409 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fe83f-e974-47aa-a968-f9b0a5ca321a None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:28.477679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fe83f-e974-47aa-a968-f9b0a5ca321a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:28.485590 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:28.489454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961fe83f-e974-47aa-a968-f9b0a5ca321a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:28.494306 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-961fe83f-e974-47aa-a968-f9b0a5ca321a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 12:01:28.494306 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:28.494306 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:28.494306 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:28.494306 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:28.494306 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:28.494306 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:28.494306 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:28.494306 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:28.494306 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:28.494306 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:28.494306 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:28.494306 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:28.494306 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:28.494306 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:28.494306 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:28.495451 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:28.495451 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:28.495451 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:28.495451 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:28.495451 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:28.495451 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:28.495451 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:28.495451 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:28.495451 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:28.495451 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:28.495451 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:28.495451 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:28.495451 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:28.495451 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:28.495451 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:28.495451 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:28.502814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:28.502814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:28.502814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:28.502814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:28.502814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:28.502814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:28.502814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:28.502814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:28.502814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:28.502814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:28.502814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 12:01:28.502814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:28.502814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 175/351] 10.222.0.98 () {56 vars in 1114 bytes} [Thu Aug 18 12:01:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:28.502814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961fe83f-e974-47aa-a968-f9b0a5ca321a None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:28.579227 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 177/352] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:01:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:01:28.583638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b50402-d6b3-4dea-aa43-4541054f6dc5 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:28.621146 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961fe83f-e974-47aa-a968-f9b0a5ca321a None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:28.662687 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd45df7-b3f4-47dc-a0a1-69e03210ee54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:28.663308 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd45df7-b3f4-47dc-a0a1-69e03210ee54 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:28.663626 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd45df7-b3f4-47dc-a0a1-69e03210ee54 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:28.663908 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd45df7-b3f4-47dc-a0a1-69e03210ee54 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:28.664588 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd45df7-b3f4-47dc-a0a1-69e03210ee54 None admin] RBAC: Adding query filter params (name=member) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:28.664902 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd45df7-b3f4-47dc-a0a1-69e03210ee54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:28.667182 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd45df7-b3f4-47dc-a0a1-69e03210ee54 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:28.677620 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 176/353] 10.222.0.98 () {56 vars in 1101 bytes} [Thu Aug 18 12:01:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:28.679587 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd45df7-b3f4-47dc-a0a1-69e03210ee54 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:28.700273 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:28.707461 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b50402-d6b3-4dea-aa43-4541054f6dc5 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:28.747822 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff66a43-c942-4fd4-93e9-359255a597ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:28.749030 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff66a43-c942-4fd4-93e9-359255a597ef None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:28.749313 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff66a43-c942-4fd4-93e9-359255a597ef None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:28.749588 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff66a43-c942-4fd4-93e9-359255a597ef None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:28.755457 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:28.763029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff66a43-c942-4fd4-93e9-359255a597ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:28.767784 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-fff66a43-c942-4fd4-93e9-359255a597ef None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 12:01:28.767784 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:28.767784 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:28.767784 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:28.767784 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:28.767784 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:28.767784 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:28.767784 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:28.767784 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:28.767784 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:28.767784 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:28.767784 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:28.767784 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:28.767784 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:28.767784 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:28.767784 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:28.772905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:28.772905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:28.772905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:28.772905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:28.772905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:28.772905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:28.772905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:28.772905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:28.772905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:28.772905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:28.772905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:28.772905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:28.772905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:28.772905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:28.772905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:28.772905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:28.773815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:28.773815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:28.773815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:28.773815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:28.773815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:28.773815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:28.773815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:28.773815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:28.773815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:28.773815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:28.773815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 18 12:01:28.773815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:28.773815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 178/354] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:01:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:28.773815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdedc624-9555-41c1-ad28-074def9db4c0 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:28.774545 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdedc624-9555-41c1-ad28-074def9db4c0 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:28.774545 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdedc624-9555-41c1-ad28-074def9db4c0 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:28.774545 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 179/355] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:28.774545 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdedc624-9555-41c1-ad28-074def9db4c0 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:28.804822 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd45df7-b3f4-47dc-a0a1-69e03210ee54 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:28.848307 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa3636d-d568-47ad-9112-d57f595bcf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:28.849683 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3636d-d568-47ad-9112-d57f595bcf7f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:28.849683 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3636d-d568-47ad-9112-d57f595bcf7f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:28.849848 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3636d-d568-47ad-9112-d57f595bcf7f None admin] PATH_INFO: `/v3/projects/dede24cb1b014e9796b44d6cdac3ed69` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:28.851455 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa3636d-d568-47ad-9112-d57f595bcf7f None admin] RBAC: Authorizing `identity:get_project(project_id=dede24cb1b014e9796b44d6cdac3ed69)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:28.852381 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa3636d-d568-47ad-9112-d57f595bcf7f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:28.854034 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 177/356] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:01:28 2022] GET /identity/v3/projects/dede24cb1b014e9796b44d6cdac3ed69 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:28.856728 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa3636d-d568-47ad-9112-d57f595bcf7f None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:28.900203 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdedc624-9555-41c1-ad28-074def9db4c0 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:28.946643 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64b5c89-5f91-4898-b902-d3b65add53d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:28.946921 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b5c89-5f91-4898-b902-d3b65add53d9 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:28.947145 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b5c89-5f91-4898-b902-d3b65add53d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:28.948449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b5c89-5f91-4898-b902-d3b65add53d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:28.949137 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64b5c89-5f91-4898-b902-d3b65add53d9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:28.949472 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64b5c89-5f91-4898-b902-d3b65add53d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:28.950911 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64b5c89-5f91-4898-b902-d3b65add53d9 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:28.963043 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 180/357] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:01:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:28.964565 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64b5c89-5f91-4898-b902-d3b65add53d9 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:28.984946 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa3636d-d568-47ad-9112-d57f595bcf7f None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:28.988129 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:29.014687 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:29.023945 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ff3c6f-4256-4e22-a867-07a89f53be01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:29.024924 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff3c6f-4256-4e22-a867-07a89f53be01 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:29.025207 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff3c6f-4256-4e22-a867-07a89f53be01 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:29.025482 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff3c6f-4256-4e22-a867-07a89f53be01 None admin] PATH_INFO: `/v3/projects/dede24cb1b014e9796b44d6cdac3ed69` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:29.026632 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ff3c6f-4256-4e22-a867-07a89f53be01 None admin] RBAC: Authorizing `identity:get_project(project_id=dede24cb1b014e9796b44d6cdac3ed69)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:29.028168 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ff3c6f-4256-4e22-a867-07a89f53be01 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:29.030310 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 178/358] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:01:28 2022] GET /identity/v3/projects/dede24cb1b014e9796b44d6cdac3ed69 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:29.031079 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ff3c6f-4256-4e22-a867-07a89f53be01 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:29.092631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64b5c89-5f91-4898-b902-d3b65add53d9 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:29.132966 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ceb50c-2799-4ef2-bab5-322f4777c09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:29.133511 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ceb50c-2799-4ef2-bab5-322f4777c09a None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:29.133833 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ceb50c-2799-4ef2-bab5-322f4777c09a None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:29.134119 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ceb50c-2799-4ef2-bab5-322f4777c09a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:29.152480 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ff3c6f-4256-4e22-a867-07a89f53be01 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:29.202208 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a266c2-ba57-4eb7-a5c6-d868b5e2be2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:29.202836 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a266c2-ba57-4eb7-a5c6-d868b5e2be2d None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:29.203314 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a266c2-ba57-4eb7-a5c6-d868b5e2be2d None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:29.203671 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a266c2-ba57-4eb7-a5c6-d868b5e2be2d None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:29.204346 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a266c2-ba57-4eb7-a5c6-d868b5e2be2d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:29.204768 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a266c2-ba57-4eb7-a5c6-d868b5e2be2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:29.206123 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a266c2-ba57-4eb7-a5c6-d868b5e2be2d None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:29.212033 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ceb50c-2799-4ef2-bab5-322f4777c09a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:29.213488 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ceb50c-2799-4ef2-bab5-322f4777c09a None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:29.216577 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 179/359] 10.222.0.98 () {56 vars in 1101 bytes} [Thu Aug 18 12:01:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:29.218575 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a6db9-e09b-4533-a7a3-d901665e4c1a None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:29.218870 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a6db9-e09b-4533-a7a3-d901665e4c1a None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:29.219151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a6db9-e09b-4533-a7a3-d901665e4c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:29.233960 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:29.237976 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-e60a6db9-e09b-4533-a7a3-d901665e4c1a None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:29.239196 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-e60a6db9-e09b-4533-a7a3-d901665e4c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:29.303872 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:29.318982 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 180/360] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:29.319870 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a6db9-e09b-4533-a7a3-d901665e4c1a None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:29.336396 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 181/361] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:01:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:01:29.337351 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ceb50c-2799-4ef2-bab5-322f4777c09a None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:29.449962 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a6db9-e09b-4533-a7a3-d901665e4c1a None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:29.473152 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ceb50c-2799-4ef2-bab5-322f4777c09a None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:29.489540 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c35281-7fa8-4c33-a4f8-75ba7e5f7cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:29.490178 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c35281-7fa8-4c33-a4f8-75ba7e5f7cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:29.490466 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c35281-7fa8-4c33-a4f8-75ba7e5f7cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:29.490753 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c35281-7fa8-4c33-a4f8-75ba7e5f7cb6 None admin] PATH_INFO: `/v3/groups/9154e4b2dc4c4aa68e28f6e3d70f783a` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:29.497153 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:29.497316 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c35281-7fa8-4c33-a4f8-75ba7e5f7cb6 None admin] RBAC: Authorizing `identity:get_group(group_id=9154e4b2dc4c4aa68e28f6e3d70f783a)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:29.499234 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c35281-7fa8-4c33-a4f8-75ba7e5f7cb6 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:29.502354 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 181/362] 10.222.0.98 () {56 vars in 1180 bytes} [Thu Aug 18 12:01:29 2022] GET /identity/v3/groups/9154e4b2dc4c4aa68e28f6e3d70f783a => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:29.503214 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c35281-7fa8-4c33-a4f8-75ba7e5f7cb6 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:29.518189 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0439e700-a57c-4dfa-91f8-52fba195a8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:29.518806 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439e700-a57c-4dfa-91f8-52fba195a8eb None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:29.519095 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439e700-a57c-4dfa-91f8-52fba195a8eb None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:29.519374 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439e700-a57c-4dfa-91f8-52fba195a8eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:29.570133 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:29.604615 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0439e700-a57c-4dfa-91f8-52fba195a8eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:29.606347 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0439e700-a57c-4dfa-91f8-52fba195a8eb None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:29.632403 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c35281-7fa8-4c33-a4f8-75ba7e5f7cb6 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:29.681850 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672b42d-7e9d-49ac-80df-ca60607e88f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:29.682811 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672b42d-7e9d-49ac-80df-ca60607e88f9 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:29.683491 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672b42d-7e9d-49ac-80df-ca60607e88f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:29.683491 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672b42d-7e9d-49ac-80df-ca60607e88f9 None admin] PATH_INFO: `/v3/groups/9154e4b2dc4c4aa68e28f6e3d70f783a` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:29.684557 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a672b42d-7e9d-49ac-80df-ca60607e88f9 None admin] RBAC: Authorizing `identity:get_group(group_id=9154e4b2dc4c4aa68e28f6e3d70f783a)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:29.686106 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a672b42d-7e9d-49ac-80df-ca60607e88f9 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:29.688713 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 182/363] 10.222.0.98 () {56 vars in 1180 bytes} [Thu Aug 18 12:01:29 2022] GET /identity/v3/groups/9154e4b2dc4c4aa68e28f6e3d70f783a => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:29.689529 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672b42d-7e9d-49ac-80df-ca60607e88f9 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:29.749857 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 182/364] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:01:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:01:29.750742 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0439e700-a57c-4dfa-91f8-52fba195a8eb None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:29.774112 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:29.801488 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672b42d-7e9d-49ac-80df-ca60607e88f9 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:29.820362 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:29.833680 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7fe0b9-58a0-4d24-8b67-99b91723b715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:29.834310 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7fe0b9-58a0-4d24-8b67-99b91723b715 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:29.834588 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7fe0b9-58a0-4d24-8b67-99b91723b715 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:29.834863 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7fe0b9-58a0-4d24-8b67-99b91723b715 None admin] PATH_INFO: `/v3/users/2e1c4363d82f4988971d67da06a6137a` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:29.836011 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7fe0b9-58a0-4d24-8b67-99b91723b715 None admin] RBAC: Authorizing `identity:get_user(user_id=2e1c4363d82f4988971d67da06a6137a)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:29.837476 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7fe0b9-58a0-4d24-8b67-99b91723b715 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:29.839028 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 183/365] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:01:29 2022] GET /identity/v3/users/2e1c4363d82f4988971d67da06a6137a => generated 293 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:29.839818 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7fe0b9-58a0-4d24-8b67-99b91723b715 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:29.860857 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0439e700-a57c-4dfa-91f8-52fba195a8eb None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:29.895373 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91abfc1-9aa7-44ee-8106-6bbfde684760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:29.896041 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91abfc1-9aa7-44ee-8106-6bbfde684760 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:29.896597 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91abfc1-9aa7-44ee-8106-6bbfde684760 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:29.896875 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91abfc1-9aa7-44ee-8106-6bbfde684760 None admin] PATH_INFO: `/v3/projects/9d8b38972ba64f509734ed17f30c88fd` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:29.898063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91abfc1-9aa7-44ee-8106-6bbfde684760 None admin] RBAC: Authorizing `identity:get_project(project_id=9d8b38972ba64f509734ed17f30c88fd)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:29.899539 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91abfc1-9aa7-44ee-8106-6bbfde684760 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:29.901165 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 183/366] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:01:29 2022] GET /identity/v3/projects/9d8b38972ba64f509734ed17f30c88fd => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:29.901937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91abfc1-9aa7-44ee-8106-6bbfde684760 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:29.938962 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7fe0b9-58a0-4d24-8b67-99b91723b715 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:29.972153 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b7b75-ec04-4f0b-8772-e55b90df6b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:29.972752 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b7b75-ec04-4f0b-8772-e55b90df6b6b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:29.973036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b7b75-ec04-4f0b-8772-e55b90df6b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:29.973306 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b7b75-ec04-4f0b-8772-e55b90df6b6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:29.974037 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75b7b75-ec04-4f0b-8772-e55b90df6b6b None admin] RBAC: Adding query filter params (group.id=9154e4b2dc4c4aa68e28f6e3d70f783a, role.id=3a4b7e53b8644e508ac440f71138be16, scope.project.id=dede24cb1b014e9796b44d6cdac3ed69) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:29.974346 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75b7b75-ec04-4f0b-8772-e55b90df6b6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:29.975799 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75b7b75-ec04-4f0b-8772-e55b90df6b6b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:29.981205 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 184/367] 10.222.0.98 () {56 vars in 1376 bytes} [Thu Aug 18 12:01:29 2022] GET /identity/v3/role_assignments?group.id=9154e4b2dc4c4aa68e28f6e3d70f783a&scope.project.id=dede24cb1b014e9796b44d6cdac3ed69&role.id=3a4b7e53b8644e508ac440f71138be16 => generated 260 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:29.981828 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b7b75-ec04-4f0b-8772-e55b90df6b6b None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:29.996492 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:30.012748 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91abfc1-9aa7-44ee-8106-6bbfde684760 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:30.048252 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:30.055937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d052590e-a625-4f1a-bd0f-b053f90cf770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:30.056656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052590e-a625-4f1a-bd0f-b053f90cf770 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:30.056945 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052590e-a625-4f1a-bd0f-b053f90cf770 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:30.057220 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052590e-a625-4f1a-bd0f-b053f90cf770 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:30.070029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d052590e-a625-4f1a-bd0f-b053f90cf770 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:30.074948 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-d052590e-a625-4f1a-bd0f-b053f90cf770 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:01:30.074948 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:30.074948 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:30.074948 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:30.074948 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:30.074948 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:30.074948 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:30.074948 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:30.074948 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:30.074948 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:30.074948 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:30.074948 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:30.074948 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:30.074948 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:30.074948 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:30.074948 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:30.075954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:30.075954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:30.075954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:30.075954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:30.075954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:30.075954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:30.075954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:30.075954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:30.075954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:30.075954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:30.075954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:30.075954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:30.075954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:30.075954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:30.075954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:30.075954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:30.076851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:30.076851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:30.076851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:30.076851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:30.076851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:30.076851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:30.076851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:30.076851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:30.076851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:30.076851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:30.076851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:01:30.076851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:30.076851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 184/368] 10.222.0.98 () {56 vars in 1096 bytes} [Thu Aug 18 12:01:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:30.077542 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d052590e-a625-4f1a-bd0f-b053f90cf770 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:30.110183 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b7b75-ec04-4f0b-8772-e55b90df6b6b None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:30.155099 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb82152-9c8b-4047-aa29-7a1eeacc7ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:30.155742 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb82152-9c8b-4047-aa29-7a1eeacc7ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:30.156023 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb82152-9c8b-4047-aa29-7a1eeacc7ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:30.156296 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb82152-9c8b-4047-aa29-7a1eeacc7ef3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:30.157025 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb82152-9c8b-4047-aa29-7a1eeacc7ef3 None admin] RBAC: Adding query filter params (group.id=9154e4b2dc4c4aa68e28f6e3d70f783a, role.id=c81e28908b9b49a38d33978df4d11716, scope.project.id=dede24cb1b014e9796b44d6cdac3ed69) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:30.157334 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb82152-9c8b-4047-aa29-7a1eeacc7ef3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:30.158766 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb82152-9c8b-4047-aa29-7a1eeacc7ef3 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:30.166880 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 185/369] 10.222.0.98 () {56 vars in 1376 bytes} [Thu Aug 18 12:01:29 2022] GET /identity/v3/role_assignments?group.id=9154e4b2dc4c4aa68e28f6e3d70f783a&scope.project.id=dede24cb1b014e9796b44d6cdac3ed69&role.id=c81e28908b9b49a38d33978df4d11716 => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:30.166880 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb82152-9c8b-4047-aa29-7a1eeacc7ef3 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:30.199300 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d052590e-a625-4f1a-bd0f-b053f90cf770 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:30.249784 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae505e-1279-49aa-9604-73a354eed5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:30.250889 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae505e-1279-49aa-9604-73a354eed5c9 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:30.251322 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae505e-1279-49aa-9604-73a354eed5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:30.252236 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae505e-1279-49aa-9604-73a354eed5c9 None admin] PATH_INFO: `/v3/projects/9d8b38972ba64f509734ed17f30c88fd` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:30.253809 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beae505e-1279-49aa-9604-73a354eed5c9 None admin] RBAC: Authorizing `identity:get_project(project_id=9d8b38972ba64f509734ed17f30c88fd)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:30.255456 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beae505e-1279-49aa-9604-73a354eed5c9 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:30.258744 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 185/370] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:01:30 2022] GET /identity/v3/projects/9d8b38972ba64f509734ed17f30c88fd => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:30.260657 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae505e-1279-49aa-9604-73a354eed5c9 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:30.287875 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:30.328775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb82152-9c8b-4047-aa29-7a1eeacc7ef3 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:30.351846 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:30.375917 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b761ee30-a854-4775-997b-52c10973deae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:30.376696 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b761ee30-a854-4775-997b-52c10973deae None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:30.377058 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b761ee30-a854-4775-997b-52c10973deae None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:30.377450 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b761ee30-a854-4775-997b-52c10973deae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:30.378261 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b761ee30-a854-4775-997b-52c10973deae None admin] RBAC: Adding query filter params (role.id=93cdab4d25f54b8ca825e7ee5a5dc2d2, user.id=2e1c4363d82f4988971d67da06a6137a, scope.system=all) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:30.378656 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b761ee30-a854-4775-997b-52c10973deae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:30.381257 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b761ee30-a854-4775-997b-52c10973deae None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:30.392506 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae505e-1279-49aa-9604-73a354eed5c9 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:30.400892 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 186/371] 10.222.0.98 () {56 vars in 1308 bytes} [Thu Aug 18 12:01:30 2022] GET /identity/v3/role_assignments?user.id=2e1c4363d82f4988971d67da06a6137a&scope.system=all&role.id=93cdab4d25f54b8ca825e7ee5a5dc2d2 => generated 510 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:30.402093 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b761ee30-a854-4775-997b-52c10973deae None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:30.452547 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff2239-7d41-45ea-a258-698c721271a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:30.453266 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff2239-7d41-45ea-a258-698c721271a2 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:30.453557 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff2239-7d41-45ea-a258-698c721271a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:30.453840 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff2239-7d41-45ea-a258-698c721271a2 None admin] PATH_INFO: `/v3/groups/9154e4b2dc4c4aa68e28f6e3d70f783a` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:30.455116 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ff2239-7d41-45ea-a258-698c721271a2 None admin] RBAC: Authorizing `identity:get_group(group_id=9154e4b2dc4c4aa68e28f6e3d70f783a)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:30.456723 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ff2239-7d41-45ea-a258-698c721271a2 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:30.463611 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 186/372] 10.222.0.98 () {56 vars in 1180 bytes} [Thu Aug 18 12:01:30 2022] GET /identity/v3/groups/9154e4b2dc4c4aa68e28f6e3d70f783a => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:30.464747 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff2239-7d41-45ea-a258-698c721271a2 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:30.564418 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:30.583564 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff2239-7d41-45ea-a258-698c721271a2 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:30.622337 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da9211-cdda-4e39-8453-1149b9cc4e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:30.623025 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da9211-cdda-4e39-8453-1149b9cc4e38 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:30.623318 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da9211-cdda-4e39-8453-1149b9cc4e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:30.623650 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da9211-cdda-4e39-8453-1149b9cc4e38 None admin] PATH_INFO: `/v3/groups/9154e4b2dc4c4aa68e28f6e3d70f783a` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:30.625108 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64da9211-cdda-4e39-8453-1149b9cc4e38 None admin] RBAC: Authorizing `identity:get_group(group_id=9154e4b2dc4c4aa68e28f6e3d70f783a)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:30.626731 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64da9211-cdda-4e39-8453-1149b9cc4e38 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:30.629466 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 187/373] 10.222.0.98 () {56 vars in 1180 bytes} [Thu Aug 18 12:01:30 2022] GET /identity/v3/groups/9154e4b2dc4c4aa68e28f6e3d70f783a => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:30.630217 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b761ee30-a854-4775-997b-52c10973deae None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:30.630316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da9211-cdda-4e39-8453-1149b9cc4e38 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:30.673622 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2194040-447e-42a2-b0b3-a0d6d1f2f7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:30.673622 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2194040-447e-42a2-b0b3-a0d6d1f2f7fc None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:30.673622 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2194040-447e-42a2-b0b3-a0d6d1f2f7fc None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:30.673622 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2194040-447e-42a2-b0b3-a0d6d1f2f7fc None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:30.674316 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2194040-447e-42a2-b0b3-a0d6d1f2f7fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:30.674316 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2194040-447e-42a2-b0b3-a0d6d1f2f7fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:30.674711 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2194040-447e-42a2-b0b3-a0d6d1f2f7fc None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:30.686361 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 187/374] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:01:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:30.687031 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2194040-447e-42a2-b0b3-a0d6d1f2f7fc None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:30.732019 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:30.753275 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da9211-cdda-4e39-8453-1149b9cc4e38 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:30.793348 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18847b0b-4b94-464b-8c44-b4ca60ac9f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:30.794138 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18847b0b-4b94-464b-8c44-b4ca60ac9f3e None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:30.794295 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18847b0b-4b94-464b-8c44-b4ca60ac9f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:30.794565 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18847b0b-4b94-464b-8c44-b4ca60ac9f3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:30.795389 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18847b0b-4b94-464b-8c44-b4ca60ac9f3e None admin] RBAC: Adding query filter params (group.id=9154e4b2dc4c4aa68e28f6e3d70f783a, role.id=c81e28908b9b49a38d33978df4d11716, scope.project.id=9d8b38972ba64f509734ed17f30c88fd) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:30.795677 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18847b0b-4b94-464b-8c44-b4ca60ac9f3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:30.797119 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18847b0b-4b94-464b-8c44-b4ca60ac9f3e None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:30.804007 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2194040-447e-42a2-b0b3-a0d6d1f2f7fc None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:30.807576 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 188/375] 10.222.0.98 () {56 vars in 1376 bytes} [Thu Aug 18 12:01:30 2022] GET /identity/v3/role_assignments?group.id=9154e4b2dc4c4aa68e28f6e3d70f783a&scope.project.id=9d8b38972ba64f509734ed17f30c88fd&role.id=c81e28908b9b49a38d33978df4d11716 => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:30.808559 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18847b0b-4b94-464b-8c44-b4ca60ac9f3e None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:30.831099 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:30.843919 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7d6f3-03d2-4210-b055-fe6a57a0779b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:30.844856 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7d6f3-03d2-4210-b055-fe6a57a0779b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:30.846229 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7d6f3-03d2-4210-b055-fe6a57a0779b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:30.846229 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7d6f3-03d2-4210-b055-fe6a57a0779b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:30.846606 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde7d6f3-03d2-4210-b055-fe6a57a0779b None admin] RBAC: Adding query filter params (group.id=9154e4b2dc4c4aa68e28f6e3d70f783a, role.id=3a4b7e53b8644e508ac440f71138be16, scope.project.id=9d8b38972ba64f509734ed17f30c88fd) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:30.847163 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde7d6f3-03d2-4210-b055-fe6a57a0779b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:30.848812 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde7d6f3-03d2-4210-b055-fe6a57a0779b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:30.855876 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 188/376] 10.222.0.98 () {56 vars in 1376 bytes} [Thu Aug 18 12:01:30 2022] GET /identity/v3/role_assignments?group.id=9154e4b2dc4c4aa68e28f6e3d70f783a&scope.project.id=9d8b38972ba64f509734ed17f30c88fd&role.id=3a4b7e53b8644e508ac440f71138be16 => generated 260 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:30.959187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18847b0b-4b94-464b-8c44-b4ca60ac9f3e None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:31.000849 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ed312-e9ec-42df-b219-f7f173293291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:31.001476 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ed312-e9ec-42df-b219-f7f173293291 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:31.001766 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ed312-e9ec-42df-b219-f7f173293291 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:31.002028 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ed312-e9ec-42df-b219-f7f173293291 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:31.080799 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ed312-e9ec-42df-b219-f7f173293291 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:31.082396 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ed312-e9ec-42df-b219-f7f173293291 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:31.098725 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:31.193224 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 189/377] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:01:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:01:31.199434 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7d6f3-03d2-4210-b055-fe6a57a0779b None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:31.308160 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:31.312085 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7d6f3-03d2-4210-b055-fe6a57a0779b None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:31.354528 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35892fe-49bd-48cf-8693-7b63b6cfee36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:31.355489 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35892fe-49bd-48cf-8693-7b63b6cfee36 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:31.355959 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35892fe-49bd-48cf-8693-7b63b6cfee36 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:31.356439 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35892fe-49bd-48cf-8693-7b63b6cfee36 None admin] PATH_INFO: `/v3/projects/c192a2eba4e24841893715c8478b559c` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:31.357969 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35892fe-49bd-48cf-8693-7b63b6cfee36 None admin] RBAC: Authorizing `identity:get_project(project_id=c192a2eba4e24841893715c8478b559c)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:31.359705 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35892fe-49bd-48cf-8693-7b63b6cfee36 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:31.361875 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 189/378] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:01:31 2022] GET /identity/v3/projects/c192a2eba4e24841893715c8478b559c => generated 344 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:31.367702 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ed312-e9ec-42df-b219-f7f173293291 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:31.487889 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ed312-e9ec-42df-b219-f7f173293291 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:31.504200 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:31.525935 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ead8f54-5117-4f8e-952b-3d588355b7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:31.526890 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead8f54-5117-4f8e-952b-3d588355b7bc None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:31.527353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead8f54-5117-4f8e-952b-3d588355b7bc None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:31.527865 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead8f54-5117-4f8e-952b-3d588355b7bc None admin] PATH_INFO: `/v3/groups/57ddb3b5c42d43ed838f1af808c71e69` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:31.534726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ead8f54-5117-4f8e-952b-3d588355b7bc None admin] RBAC: Authorizing `identity:get_group(group_id=57ddb3b5c42d43ed838f1af808c71e69)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:31.536473 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ead8f54-5117-4f8e-952b-3d588355b7bc None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:31.538351 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 190/379] 10.222.0.98 () {56 vars in 1180 bytes} [Thu Aug 18 12:01:31 2022] GET /identity/v3/groups/57ddb3b5c42d43ed838f1af808c71e69 => generated 229 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:31.544827 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35892fe-49bd-48cf-8693-7b63b6cfee36 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:31.724372 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35892fe-49bd-48cf-8693-7b63b6cfee36 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:31.762842 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e779c81-4665-40a2-861b-5aa03bda99d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:31.763578 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e779c81-4665-40a2-861b-5aa03bda99d6 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:31.764426 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e779c81-4665-40a2-861b-5aa03bda99d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:31.764710 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e779c81-4665-40a2-861b-5aa03bda99d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:31.765449 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e779c81-4665-40a2-861b-5aa03bda99d6 None admin] RBAC: Adding query filter params (group.id=57ddb3b5c42d43ed838f1af808c71e69, role.id=c53a0e13b58642f291c856efe1f0b87d, scope.project.id=c192a2eba4e24841893715c8478b559c) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:31.765764 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e779c81-4665-40a2-861b-5aa03bda99d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:31.767201 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e779c81-4665-40a2-861b-5aa03bda99d6 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:31.775928 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 190/380] 10.222.0.98 () {56 vars in 1376 bytes} [Thu Aug 18 12:01:31 2022] GET /identity/v3/role_assignments?group.id=57ddb3b5c42d43ed838f1af808c71e69&scope.project.id=c192a2eba4e24841893715c8478b559c&role.id=c53a0e13b58642f291c856efe1f0b87d => generated 260 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:32.190744 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e98608-7c30-4eb6-89cb-7ebcd313af12 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:32.191779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e98608-7c30-4eb6-89cb-7ebcd313af12 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:32.192262 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e98608-7c30-4eb6-89cb-7ebcd313af12 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:32.194015 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 191/381] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:32.199730 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327fb4d-7a2d-4000-a861-ca3136a47d5d None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:32.200333 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327fb4d-7a2d-4000-a861-ca3136a47d5d None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:32.200814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327fb4d-7a2d-4000-a861-ca3136a47d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:32.224191 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-3327fb4d-7a2d-4000-a861-ca3136a47d5d None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:32.225726 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-3327fb4d-7a2d-4000-a861-ca3136a47d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:32.265751 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:32.320122 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 191/382] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 12:01:32.592990 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55cff1-5a53-4fdb-aed9-b1cc303f50f3 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:32.593689 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55cff1-5a53-4fdb-aed9-b1cc303f50f3 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:32.594173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55cff1-5a53-4fdb-aed9-b1cc303f50f3 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:32.596148 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 192/383] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:32.601360 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c05c2-a682-48cd-82eb-52d33e223e43 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:32.601908 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c05c2-a682-48cd-82eb-52d33e223e43 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:32.602428 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c05c2-a682-48cd-82eb-52d33e223e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:32.627247 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-815c05c2-a682-48cd-82eb-52d33e223e43 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:32.628762 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-815c05c2-a682-48cd-82eb-52d33e223e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:32.721876 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 192/384] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:32.755466 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55cff1-5a53-4fdb-aed9-b1cc303f50f3 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:32.909622 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55cff1-5a53-4fdb-aed9-b1cc303f50f3 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:32.961968 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba99f0e-e370-43df-b0da-09161316a86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:32.962661 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba99f0e-e370-43df-b0da-09161316a86c None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:32.962947 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba99f0e-e370-43df-b0da-09161316a86c None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:32.963218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba99f0e-e370-43df-b0da-09161316a86c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:32.977576 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba99f0e-e370-43df-b0da-09161316a86c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:32.982886 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-4ba99f0e-e370-43df-b0da-09161316a86c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 12:01:32.982886 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:32.982886 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:32.982886 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:32.982886 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:32.982886 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:32.982886 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:32.982886 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:32.982886 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:32.982886 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:32.982886 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:32.982886 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:32.982886 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:32.982886 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:32.982886 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:32.982886 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:32.983845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:32.983845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:32.983845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:32.983845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:32.983845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:32.983845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:32.983845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:32.983845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:32.983845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:32.983845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:32.983845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:32.983845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:32.983845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:32.983845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:32.983845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:32.983845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:32.985070 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:32.985070 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:32.985070 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:32.985070 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:32.985070 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:32.985070 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:32.985070 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:32.985070 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:32.985070 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:32.985070 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:32.985070 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 12:01:32.985070 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:32.985070 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 193/385] 10.222.0.98 () {56 vars in 1114 bytes} [Thu Aug 18 12:01:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:01:32.990023 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c05c2-a682-48cd-82eb-52d33e223e43 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:33.110940 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:33.120555 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c05c2-a682-48cd-82eb-52d33e223e43 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:33.168653 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ded00-6592-43de-9d53-9aabcd3546db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:33.169180 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ded00-6592-43de-9d53-9aabcd3546db None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:33.169459 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ded00-6592-43de-9d53-9aabcd3546db None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:33.169731 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ded00-6592-43de-9d53-9aabcd3546db None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:33.170434 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ded00-6592-43de-9d53-9aabcd3546db None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:33.170735 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ded00-6592-43de-9d53-9aabcd3546db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:33.172125 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ded00-6592-43de-9d53-9aabcd3546db None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:33.185721 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba99f0e-e370-43df-b0da-09161316a86c None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:33.199580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 193/386] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:01:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:33.207380 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ded00-6592-43de-9d53-9aabcd3546db None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:33.267405 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:33.367262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ded00-6592-43de-9d53-9aabcd3546db None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:33.378343 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba99f0e-e370-43df-b0da-09161316a86c None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:33.413097 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35a96f-65f5-4b96-99da-9dfe920465f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:33.414137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35a96f-65f5-4b96-99da-9dfe920465f6 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:33.414672 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35a96f-65f5-4b96-99da-9dfe920465f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:33.415148 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35a96f-65f5-4b96-99da-9dfe920465f6 None admin] PATH_INFO: `/v3/groups/9154e4b2dc4c4aa68e28f6e3d70f783a` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:33.416995 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e35a96f-65f5-4b96-99da-9dfe920465f6 None admin] RBAC: Authorizing `identity:get_group(group_id=9154e4b2dc4c4aa68e28f6e3d70f783a)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:33.418651 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e35a96f-65f5-4b96-99da-9dfe920465f6 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:33.427681 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 194/387] 10.222.0.98 () {56 vars in 1180 bytes} [Thu Aug 18 12:01:33 2022] GET /identity/v3/groups/9154e4b2dc4c4aa68e28f6e3d70f783a => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:33.430115 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19def085-df71-4181-8100-a8541f4e5e97 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:33.430493 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19def085-df71-4181-8100-a8541f4e5e97 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:33.430790 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19def085-df71-4181-8100-a8541f4e5e97 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:33.432347 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 195/388] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:33.433109 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19def085-df71-4181-8100-a8541f4e5e97 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:33.448628 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe24a03c-ca94-4a07-ba3a-eb1a25fab565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:33.450181 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24a03c-ca94-4a07-ba3a-eb1a25fab565 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:33.450758 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24a03c-ca94-4a07-ba3a-eb1a25fab565 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:33.451241 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24a03c-ca94-4a07-ba3a-eb1a25fab565 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:33.466111 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe24a03c-ca94-4a07-ba3a-eb1a25fab565 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:33.466729 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:33.473971 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-fe24a03c-ca94-4a07-ba3a-eb1a25fab565 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 12:01:33.473971 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:33.473971 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:33.473971 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:33.473971 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:33.473971 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:33.473971 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:33.473971 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:33.473971 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:33.473971 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:33.473971 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:33.473971 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:33.473971 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:33.473971 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:33.473971 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:33.473971 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:33.483501 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:33.483501 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:33.483501 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:33.483501 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:33.483501 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:33.483501 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:33.483501 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:33.483501 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:33.483501 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:33.483501 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:33.483501 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:33.483501 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:33.483501 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:33.483501 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:33.483501 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:33.483501 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:33.484384 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:33.484384 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:33.484384 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:33.484384 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:33.484384 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:33.484384 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:33.484384 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:33.484384 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:33.484384 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:33.484384 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:33.484384 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 12:01:33.484384 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:33.484384 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 194/389] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:01:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:01:33.484384 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163ca74-29ec-412c-836c-a9348717d368 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:33.485170 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163ca74-29ec-412c-836c-a9348717d368 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:33.485170 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163ca74-29ec-412c-836c-a9348717d368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:33.497950 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-5163ca74-29ec-412c-836c-a9348717d368 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:33.500320 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-5163ca74-29ec-412c-836c-a9348717d368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:33.555640 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19def085-df71-4181-8100-a8541f4e5e97 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:33.570047 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 195/390] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:33.571635 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd61c1e-3a1b-4024-9f46-1e3986edc755 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:33.572110 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd61c1e-3a1b-4024-9f46-1e3986edc755 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:33.572565 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd61c1e-3a1b-4024-9f46-1e3986edc755 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:33.574250 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 196/391] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:33.575183 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd61c1e-3a1b-4024-9f46-1e3986edc755 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:33.606730 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af15879-84d5-4cfe-8626-13b10df6577b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:33.608486 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af15879-84d5-4cfe-8626-13b10df6577b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:33.609261 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af15879-84d5-4cfe-8626-13b10df6577b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:33.609730 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af15879-84d5-4cfe-8626-13b10df6577b None admin] PATH_INFO: `/v3/projects/dede24cb1b014e9796b44d6cdac3ed69` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:33.612054 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af15879-84d5-4cfe-8626-13b10df6577b None admin] RBAC: Authorizing `identity:get_project(project_id=dede24cb1b014e9796b44d6cdac3ed69)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:33.614608 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:33.615780 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af15879-84d5-4cfe-8626-13b10df6577b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:33.622875 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 196/392] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:01:33 2022] GET /identity/v3/projects/dede24cb1b014e9796b44d6cdac3ed69 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:33.622875 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c5547-981b-4c9c-9989-42ff5a24aa9c None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:33.622875 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c5547-981b-4c9c-9989-42ff5a24aa9c None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:33.622875 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c5547-981b-4c9c-9989-42ff5a24aa9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:33.647827 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-0a1c5547-981b-4c9c-9989-42ff5a24aa9c None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:33.649016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-0a1c5547-981b-4c9c-9989-42ff5a24aa9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:33.730490 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd61c1e-3a1b-4024-9f46-1e3986edc755 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:33.739869 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 197/393] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:33.747911 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c5547-981b-4c9c-9989-42ff5a24aa9c None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:33.774273 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278d7bd-f056-44e1-bc89-4ec137b757fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:33.775263 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278d7bd-f056-44e1-bc89-4ec137b757fa None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:33.780861 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278d7bd-f056-44e1-bc89-4ec137b757fa None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:33.780861 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278d7bd-f056-44e1-bc89-4ec137b757fa None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:33.780861 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c278d7bd-f056-44e1-bc89-4ec137b757fa None admin] RBAC: Adding query filter params (name=member) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:33.781113 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:33.781187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c278d7bd-f056-44e1-bc89-4ec137b757fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:33.781187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c278d7bd-f056-44e1-bc89-4ec137b757fa None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:33.790538 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 197/394] 10.222.0.98 () {56 vars in 1101 bytes} [Thu Aug 18 12:01:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:33.800543 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278d7bd-f056-44e1-bc89-4ec137b757fa None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:33.908176 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c5547-981b-4c9c-9989-42ff5a24aa9c None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:33.949425 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:33.953266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d071e07-20b8-49f7-8f59-c0502d9f82c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:33.954028 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d071e07-20b8-49f7-8f59-c0502d9f82c5 None admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:33.954405 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d071e07-20b8-49f7-8f59-c0502d9f82c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:33.954911 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d071e07-20b8-49f7-8f59-c0502d9f82c5 None admin] PATH_INFO: `/v3/projects/dede24cb1b014e9796b44d6cdac3ed69/groups/9154e4b2dc4c4aa68e28f6e3d70f783a/roles/3a4b7e53b8644e508ac440f71138be16` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:33.958010 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d071e07-20b8-49f7-8f59-c0502d9f82c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=dede24cb1b014e9796b44d6cdac3ed69, group_id=9154e4b2dc4c4aa68e28f6e3d70f783a, role_id=3a4b7e53b8644e508ac440f71138be16)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:33.960162 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d071e07-20b8-49f7-8f59-c0502d9f82c5 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:33.961734 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278d7bd-f056-44e1-bc89-4ec137b757fa None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:33.998249 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9114ffd-d12f-44c6-bdbf-ed7fa711f20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:33.999198 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9114ffd-d12f-44c6-bdbf-ed7fa711f20f None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:33.999722 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9114ffd-d12f-44c6-bdbf-ed7fa711f20f None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:34.000201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9114ffd-d12f-44c6-bdbf-ed7fa711f20f None admin] PATH_INFO: `/v3/groups/9154e4b2dc4c4aa68e28f6e3d70f783a` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:34.001568 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9114ffd-d12f-44c6-bdbf-ed7fa711f20f None admin] RBAC: Authorizing `identity:get_group(group_id=9154e4b2dc4c4aa68e28f6e3d70f783a)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:34.003237 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9114ffd-d12f-44c6-bdbf-ed7fa711f20f None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:34.005352 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 198/395] 10.222.0.98 () {56 vars in 1180 bytes} [Thu Aug 18 12:01:33 2022] GET /identity/v3/groups/9154e4b2dc4c4aa68e28f6e3d70f783a => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:34.006347 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9114ffd-d12f-44c6-bdbf-ed7fa711f20f None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:34.051908 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 198/396] 10.222.0.98 () {58 vars in 1442 bytes} [Thu Aug 18 12:01:33 2022] PUT /identity/v3/projects/dede24cb1b014e9796b44d6cdac3ed69/groups/9154e4b2dc4c4aa68e28f6e3d70f783a/roles/3a4b7e53b8644e508ac440f71138be16 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:01:34.056629 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d071e07-20b8-49f7-8f59-c0502d9f82c5 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:34.135163 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9114ffd-d12f-44c6-bdbf-ed7fa711f20f None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:34.155941 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:34.175736 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3d8fd-b90f-4334-9ac6-e868af345a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:34.176377 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3d8fd-b90f-4334-9ac6-e868af345a39 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:34.176652 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3d8fd-b90f-4334-9ac6-e868af345a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:34.176924 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3d8fd-b90f-4334-9ac6-e868af345a39 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:34.178513 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d071e07-20b8-49f7-8f59-c0502d9f82c5 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:34.189940 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d3d8fd-b90f-4334-9ac6-e868af345a39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:34.194839 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-b9d3d8fd-b90f-4334-9ac6-e868af345a39 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 12:01:34.194839 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:34.194839 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:34.194839 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:34.194839 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:34.194839 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:34.194839 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:34.194839 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:34.194839 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:34.194839 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:34.194839 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:34.194839 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:34.194839 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:34.194839 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:34.194839 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:34.194839 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:34.195970 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:34.195970 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:34.195970 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:34.195970 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:34.195970 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:34.195970 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:34.195970 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:34.195970 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:34.195970 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:34.195970 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:34.195970 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:34.195970 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:34.195970 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:34.195970 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:34.195970 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:34.195970 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:34.196982 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:34.196982 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:34.196982 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:34.196982 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:34.196982 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:34.196982 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:34.196982 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:34.196982 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:34.196982 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:34.196982 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:34.196982 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 12:01:34.196982 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:34.196982 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 199/397] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:01:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:34.196982 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3d8fd-b90f-4334-9ac6-e868af345a39 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:34.224091 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf87314-4668-4a7c-8f24-fcf267e7e4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:34.224460 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf87314-4668-4a7c-8f24-fcf267e7e4c4 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:34.224713 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:34.224844 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf87314-4668-4a7c-8f24-fcf267e7e4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:34.225031 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf87314-4668-4a7c-8f24-fcf267e7e4c4 None admin] PATH_INFO: `/v3/projects/dede24cb1b014e9796b44d6cdac3ed69` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:34.226166 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf87314-4668-4a7c-8f24-fcf267e7e4c4 None admin] RBAC: Authorizing `identity:get_project(project_id=dede24cb1b014e9796b44d6cdac3ed69)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:34.228180 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf87314-4668-4a7c-8f24-fcf267e7e4c4 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:34.233957 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 199/398] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:01:34 2022] GET /identity/v3/projects/dede24cb1b014e9796b44d6cdac3ed69 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:34.233957 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100ab0f-931e-4600-b700-038979afc41b None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:34.233957 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100ab0f-931e-4600-b700-038979afc41b None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:34.233957 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100ab0f-931e-4600-b700-038979afc41b None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:34.233957 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 200/399] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:34.233957 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f100ab0f-931e-4600-b700-038979afc41b None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:34.315943 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3d8fd-b90f-4334-9ac6-e868af345a39 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:34.346494 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f100ab0f-931e-4600-b700-038979afc41b None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:34.350788 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b0d9a3-4e5b-4f73-9528-a11a27c55c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:34.351442 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0d9a3-4e5b-4f73-9528-a11a27c55c47 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:34.351731 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0d9a3-4e5b-4f73-9528-a11a27c55c47 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:34.352012 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0d9a3-4e5b-4f73-9528-a11a27c55c47 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:34.363166 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b0d9a3-4e5b-4f73-9528-a11a27c55c47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:34.368304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-47b0d9a3-4e5b-4f73-9528-a11a27c55c47 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 12:01:34.368304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:34.368304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:34.368304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:34.368304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:34.368304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:34.368304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:34.368304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:34.368304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:34.368304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:34.368304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:34.368304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:34.368304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:34.368304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:34.368304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:34.368304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:34.369201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:34.369201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:34.369201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:34.369201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:34.369201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:34.369201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:34.369201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:34.369201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:34.369201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:34.369201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:34.369201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:34.369201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:34.369201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:34.369201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:34.369201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:34.369201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:34.370028 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:34.370028 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:34.370028 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:34.370028 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:34.370028 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:34.370028 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:34.370028 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:34.370028 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:34.370028 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:34.370028 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:34.370028 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 12:01:34.370028 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:34.370028 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 200/400] 10.222.0.98 () {56 vars in 1114 bytes} [Thu Aug 18 12:01:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:34.371344 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b0d9a3-4e5b-4f73-9528-a11a27c55c47 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:34.387366 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b8d04-7131-4f5a-a0c5-90b17aee036d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:34.388000 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b8d04-7131-4f5a-a0c5-90b17aee036d None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:34.388284 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b8d04-7131-4f5a-a0c5-90b17aee036d None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:34.388559 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b8d04-7131-4f5a-a0c5-90b17aee036d None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:34.389981 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b8d04-7131-4f5a-a0c5-90b17aee036d None admin] RBAC: Adding query filter params (name=member) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:34.390289 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b8d04-7131-4f5a-a0c5-90b17aee036d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:34.391688 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b8d04-7131-4f5a-a0c5-90b17aee036d None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:34.400564 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 201/401] 10.222.0.98 () {56 vars in 1101 bytes} [Thu Aug 18 12:01:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:34.401768 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c3281-ad7a-488d-b371-53cbbca3a507 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:34.402054 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c3281-ad7a-488d-b371-53cbbca3a507 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:34.402328 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c3281-ad7a-488d-b371-53cbbca3a507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:34.420830 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-5a3c3281-ad7a-488d-b371-53cbbca3a507 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:34.422013 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-5a3c3281-ad7a-488d-b371-53cbbca3a507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:34.424860 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:34.464374 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:34.489009 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b0d9a3-4e5b-4f73-9528-a11a27c55c47 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:34.491892 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 202/402] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:34.492470 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3c3281-ad7a-488d-b371-53cbbca3a507 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:34.540381 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e8379e-7ffd-4cfa-90a4-dab804c716b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:34.540978 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8379e-7ffd-4cfa-90a4-dab804c716b8 None admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:34.541313 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8379e-7ffd-4cfa-90a4-dab804c716b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:34.541588 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8379e-7ffd-4cfa-90a4-dab804c716b8 None admin] PATH_INFO: `/v3/projects/dede24cb1b014e9796b44d6cdac3ed69/groups/9154e4b2dc4c4aa68e28f6e3d70f783a/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:34.543648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e8379e-7ffd-4cfa-90a4-dab804c716b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=dede24cb1b014e9796b44d6cdac3ed69, group_id=9154e4b2dc4c4aa68e28f6e3d70f783a, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:34.545192 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e8379e-7ffd-4cfa-90a4-dab804c716b8 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:34.609307 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3c3281-ad7a-488d-b371-53cbbca3a507 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:34.631038 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 201/403] 10.222.0.98 () {58 vars in 1442 bytes} [Thu Aug 18 12:01:34 2022] PUT /identity/v3/projects/dede24cb1b014e9796b44d6cdac3ed69/groups/9154e4b2dc4c4aa68e28f6e3d70f783a/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:01:34.635848 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e8379e-7ffd-4cfa-90a4-dab804c716b8 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:34.726154 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccaf35a-7818-4b53-9531-ff7a296dce62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:34.727173 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccaf35a-7818-4b53-9531-ff7a296dce62 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:34.727686 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccaf35a-7818-4b53-9531-ff7a296dce62 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:34.728148 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccaf35a-7818-4b53-9531-ff7a296dce62 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:34.729343 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccaf35a-7818-4b53-9531-ff7a296dce62 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:34.729858 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccaf35a-7818-4b53-9531-ff7a296dce62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:34.732202 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccaf35a-7818-4b53-9531-ff7a296dce62 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:34.770674 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 203/404] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:01:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:34.772355 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccaf35a-7818-4b53-9531-ff7a296dce62 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:34.780478 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:34.891316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e8379e-7ffd-4cfa-90a4-dab804c716b8 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:34.896876 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccaf35a-7818-4b53-9531-ff7a296dce62 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:34.934321 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9b6e4-ad0a-4c63-81f4-3488656e88f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:34.934714 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7853a-1947-45c2-8aee-069f3df9b306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:34.934966 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9b6e4-ad0a-4c63-81f4-3488656e88f5 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:34.935249 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9b6e4-ad0a-4c63-81f4-3488656e88f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:34.935361 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7853a-1947-45c2-8aee-069f3df9b306 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:34.935554 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9b6e4-ad0a-4c63-81f4-3488656e88f5 None admin] PATH_INFO: `/v3/groups/9154e4b2dc4c4aa68e28f6e3d70f783a` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:34.935662 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7853a-1947-45c2-8aee-069f3df9b306 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:34.935864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7853a-1947-45c2-8aee-069f3df9b306 None admin] PATH_INFO: `/v3/groups/9154e4b2dc4c4aa68e28f6e3d70f783a` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:34.937042 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d7853a-1947-45c2-8aee-069f3df9b306 None admin] RBAC: Authorizing `identity:get_group(group_id=9154e4b2dc4c4aa68e28f6e3d70f783a)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:34.937769 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c9b6e4-ad0a-4c63-81f4-3488656e88f5 None admin] RBAC: Authorizing `identity:get_group(group_id=9154e4b2dc4c4aa68e28f6e3d70f783a)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:34.938575 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d7853a-1947-45c2-8aee-069f3df9b306 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:34.939473 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c9b6e4-ad0a-4c63-81f4-3488656e88f5 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:34.940962 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 204/406] 10.222.0.98 () {56 vars in 1180 bytes} [Thu Aug 18 12:01:34 2022] GET /identity/v3/groups/9154e4b2dc4c4aa68e28f6e3d70f783a => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:34.941616 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9b6e4-ad0a-4c63-81f4-3488656e88f5 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:34.943504 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 202/406] 10.222.0.98 () {56 vars in 1180 bytes} [Thu Aug 18 12:01:34 2022] GET /identity/v3/groups/9154e4b2dc4c4aa68e28f6e3d70f783a => generated 226 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:34.946212 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7853a-1947-45c2-8aee-069f3df9b306 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:34.975807 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:35.020267 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:35.052717 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9b6e4-ad0a-4c63-81f4-3488656e88f5 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:35.087225 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7853a-1947-45c2-8aee-069f3df9b306 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:35.089334 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87939bb3-32b7-408f-8c3b-a90c14d640d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:35.089972 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87939bb3-32b7-408f-8c3b-a90c14d640d5 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:35.090253 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87939bb3-32b7-408f-8c3b-a90c14d640d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:35.090529 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87939bb3-32b7-408f-8c3b-a90c14d640d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:35.104835 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87939bb3-32b7-408f-8c3b-a90c14d640d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:35.110867 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-87939bb3-32b7-408f-8c3b-a90c14d640d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:01:35.110867 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:35.110867 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:35.110867 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:35.110867 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:35.110867 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:35.110867 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:35.110867 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:35.110867 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:35.110867 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:35.110867 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:35.110867 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:35.110867 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:35.110867 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:35.110867 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:35.110867 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:35.111853 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:35.111853 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:35.111853 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:35.111853 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:35.111853 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:35.111853 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:35.111853 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:35.111853 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:35.111853 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:35.111853 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:35.111853 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:35.111853 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:35.111853 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:35.111853 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:35.111853 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:35.111853 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:35.112668 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:35.112668 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:35.112668 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:35.112668 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:35.112668 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:35.112668 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:35.112668 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:35.112668 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:35.112668 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:35.112668 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:35.112668 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:01:35.112668 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:35.112668 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 205/407] 10.222.0.98 () {56 vars in 1096 bytes} [Thu Aug 18 12:01:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:35.113337 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87939bb3-32b7-408f-8c3b-a90c14d640d5 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:35.130141 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fc4f39-a51f-4134-8602-994e1dba8b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:35.130831 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc4f39-a51f-4134-8602-994e1dba8b12 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:35.131120 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc4f39-a51f-4134-8602-994e1dba8b12 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:35.131399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc4f39-a51f-4134-8602-994e1dba8b12 None admin] PATH_INFO: `/v3/projects/9d8b38972ba64f509734ed17f30c88fd` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:35.132573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fc4f39-a51f-4134-8602-994e1dba8b12 None admin] RBAC: Authorizing `identity:get_project(project_id=9d8b38972ba64f509734ed17f30c88fd)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:35.134024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fc4f39-a51f-4134-8602-994e1dba8b12 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:35.140667 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 203/408] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:01:34 2022] GET /identity/v3/projects/9d8b38972ba64f509734ed17f30c88fd => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:35.141573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fc4f39-a51f-4134-8602-994e1dba8b12 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:35.223683 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:35.224332 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87939bb3-32b7-408f-8c3b-a90c14d640d5 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:35.252925 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fc4f39-a51f-4134-8602-994e1dba8b12 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:35.265430 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00882dab-f02c-4342-b10d-0acaad1a1dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:35.266081 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00882dab-f02c-4342-b10d-0acaad1a1dd9 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:35.266366 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00882dab-f02c-4342-b10d-0acaad1a1dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:35.266643 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00882dab-f02c-4342-b10d-0acaad1a1dd9 None admin] PATH_INFO: `/v3/projects/9d8b38972ba64f509734ed17f30c88fd` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:35.267991 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00882dab-f02c-4342-b10d-0acaad1a1dd9 None admin] RBAC: Authorizing `identity:get_project(project_id=9d8b38972ba64f509734ed17f30c88fd)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:35.269546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00882dab-f02c-4342-b10d-0acaad1a1dd9 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:35.271555 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 206/409] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:01:35 2022] GET /identity/v3/projects/9d8b38972ba64f509734ed17f30c88fd => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:35.272661 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00882dab-f02c-4342-b10d-0acaad1a1dd9 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:35.276578 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:35.291765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b540b-db17-41a3-ad5c-3484010678a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:35.292387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b540b-db17-41a3-ad5c-3484010678a3 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:35.292673 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b540b-db17-41a3-ad5c-3484010678a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:35.292955 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b540b-db17-41a3-ad5c-3484010678a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:35.293619 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868b540b-db17-41a3-ad5c-3484010678a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:35.293927 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868b540b-db17-41a3-ad5c-3484010678a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:35.295866 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868b540b-db17-41a3-ad5c-3484010678a3 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:35.309087 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 204/410] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:01:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:35.310244 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b540b-db17-41a3-ad5c-3484010678a3 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:35.381144 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00882dab-f02c-4342-b10d-0acaad1a1dd9 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:35.419646 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2cc2ad-29ae-4e7f-91f8-7d6bc44585b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:35.420384 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2cc2ad-29ae-4e7f-91f8-7d6bc44585b5 None admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:35.420667 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2cc2ad-29ae-4e7f-91f8-7d6bc44585b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:35.420785 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b540b-db17-41a3-ad5c-3484010678a3 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:35.421205 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2cc2ad-29ae-4e7f-91f8-7d6bc44585b5 None admin] PATH_INFO: `/v3/projects/9d8b38972ba64f509734ed17f30c88fd/groups/9154e4b2dc4c4aa68e28f6e3d70f783a/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:35.422948 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2cc2ad-29ae-4e7f-91f8-7d6bc44585b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=9d8b38972ba64f509734ed17f30c88fd, group_id=9154e4b2dc4c4aa68e28f6e3d70f783a, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:35.424528 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2cc2ad-29ae-4e7f-91f8-7d6bc44585b5 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:35.456564 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15603c-bfe2-4af8-93a4-be6e23fb3a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:35.457234 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15603c-bfe2-4af8-93a4-be6e23fb3a40 None admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:35.457521 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15603c-bfe2-4af8-93a4-be6e23fb3a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:35.457795 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15603c-bfe2-4af8-93a4-be6e23fb3a40 None admin] PATH_INFO: `/v3/projects/9d8b38972ba64f509734ed17f30c88fd/groups/9154e4b2dc4c4aa68e28f6e3d70f783a/roles/3a4b7e53b8644e508ac440f71138be16` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:35.460009 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf15603c-bfe2-4af8-93a4-be6e23fb3a40 None admin] RBAC: Authorizing `identity:create_grant(project_id=9d8b38972ba64f509734ed17f30c88fd, group_id=9154e4b2dc4c4aa68e28f6e3d70f783a, role_id=3a4b7e53b8644e508ac440f71138be16)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:35.461541 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf15603c-bfe2-4af8-93a4-be6e23fb3a40 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:35.465399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:35.503157 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 207/411] 10.222.0.98 () {58 vars in 1442 bytes} [Thu Aug 18 12:01:35 2022] PUT /identity/v3/projects/9d8b38972ba64f509734ed17f30c88fd/groups/9154e4b2dc4c4aa68e28f6e3d70f783a/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:01:35.504115 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2cc2ad-29ae-4e7f-91f8-7d6bc44585b5 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:35.531258 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 205/412] 10.222.0.98 () {58 vars in 1442 bytes} [Thu Aug 18 12:01:35 2022] PUT /identity/v3/projects/9d8b38972ba64f509734ed17f30c88fd/groups/9154e4b2dc4c4aa68e28f6e3d70f783a/roles/3a4b7e53b8644e508ac440f71138be16 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:01:35.593887 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:35.643535 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2cc2ad-29ae-4e7f-91f8-7d6bc44585b5 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:35.699610 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c9f2b5-eefb-4a47-8034-799237454e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:35.700276 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c9f2b5-eefb-4a47-8034-799237454e79 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:35.700557 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c9f2b5-eefb-4a47-8034-799237454e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:35.700906 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c9f2b5-eefb-4a47-8034-799237454e79 None admin] PATH_INFO: `/v3/groups/57ddb3b5c42d43ed838f1af808c71e69` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:35.704645 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c9f2b5-eefb-4a47-8034-799237454e79 None admin] RBAC: Authorizing `identity:get_group(group_id=57ddb3b5c42d43ed838f1af808c71e69)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:35.706196 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c9f2b5-eefb-4a47-8034-799237454e79 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:35.708304 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 208/413] 10.222.0.98 () {56 vars in 1180 bytes} [Thu Aug 18 12:01:35 2022] GET /identity/v3/groups/57ddb3b5c42d43ed838f1af808c71e69 => generated 229 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:35.714258 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15603c-bfe2-4af8-93a4-be6e23fb3a40 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:35.847358 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15603c-bfe2-4af8-93a4-be6e23fb3a40 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:35.894594 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4bff64-19bd-4909-bf81-5aa643715f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:35.902751 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4bff64-19bd-4909-bf81-5aa643715f47 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:35.902751 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4bff64-19bd-4909-bf81-5aa643715f47 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:35.902751 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4bff64-19bd-4909-bf81-5aa643715f47 None admin] PATH_INFO: `/v3/projects/c192a2eba4e24841893715c8478b559c` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:35.902751 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4bff64-19bd-4909-bf81-5aa643715f47 None admin] RBAC: Authorizing `identity:get_project(project_id=c192a2eba4e24841893715c8478b559c)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:35.902751 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4bff64-19bd-4909-bf81-5aa643715f47 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:35.902751 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 206/414] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:01:35 2022] GET /identity/v3/projects/c192a2eba4e24841893715c8478b559c => generated 344 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:35.906620 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c9f2b5-eefb-4a47-8034-799237454e79 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:36.020944 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c9f2b5-eefb-4a47-8034-799237454e79 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:36.049184 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:36.059028 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e67326-c2d2-4a26-abef-f5e37b227564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:36.061485 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e67326-c2d2-4a26-abef-f5e37b227564 None admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:36.061485 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e67326-c2d2-4a26-abef-f5e37b227564 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:36.062004 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e67326-c2d2-4a26-abef-f5e37b227564 None admin] PATH_INFO: `/v3/projects/c192a2eba4e24841893715c8478b559c/groups/57ddb3b5c42d43ed838f1af808c71e69/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:36.064213 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e67326-c2d2-4a26-abef-f5e37b227564 None admin] RBAC: Authorizing `identity:create_grant(project_id=c192a2eba4e24841893715c8478b559c, group_id=57ddb3b5c42d43ed838f1af808c71e69, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:36.067057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e67326-c2d2-4a26-abef-f5e37b227564 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:36.149190 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 209/415] 10.222.0.98 () {58 vars in 1442 bytes} [Thu Aug 18 12:01:35 2022] PUT /identity/v3/projects/c192a2eba4e24841893715c8478b559c/groups/57ddb3b5c42d43ed838f1af808c71e69/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:01:36.207854 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca43a5-45c1-4452-a2d4-dee402c1dd93 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:36.208524 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca43a5-45c1-4452-a2d4-dee402c1dd93 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:36.209029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca43a5-45c1-4452-a2d4-dee402c1dd93 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:36.210686 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 207/416] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:36.216818 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22984c7-134a-49aa-bda4-0bcfea0ff20c None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:36.217478 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22984c7-134a-49aa-bda4-0bcfea0ff20c None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:36.217970 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22984c7-134a-49aa-bda4-0bcfea0ff20c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:36.238261 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-c22984c7-134a-49aa-bda4-0bcfea0ff20c None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:36.239943 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-c22984c7-134a-49aa-bda4-0bcfea0ff20c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:36.317481 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 210/417] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:36.601297 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca43a5-45c1-4452-a2d4-dee402c1dd93 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:36.689592 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:36.730285 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca43a5-45c1-4452-a2d4-dee402c1dd93 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:36.765875 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a791a577-ec90-4431-bcfa-48fed43dc904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:36.766528 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791a577-ec90-4431-bcfa-48fed43dc904 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:36.766849 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791a577-ec90-4431-bcfa-48fed43dc904 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:36.767135 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791a577-ec90-4431-bcfa-48fed43dc904 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:36.778470 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a791a577-ec90-4431-bcfa-48fed43dc904 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:36.784555 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-a791a577-ec90-4431-bcfa-48fed43dc904 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 12:01:36.784555 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:36.784555 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:36.784555 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:36.784555 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:36.784555 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:36.784555 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:36.784555 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:36.784555 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:36.784555 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:36.784555 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:36.784555 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:36.784555 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:36.784555 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:36.784555 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:36.784555 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:36.785460 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:36.785460 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:36.785460 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:36.785460 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:36.785460 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:36.785460 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:36.785460 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:36.785460 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:36.785460 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:36.785460 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:36.785460 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:36.785460 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:36.785460 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:36.785460 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:36.785460 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:36.785460 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:36.786259 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:36.786259 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:36.786259 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:36.786259 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:36.786259 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:36.786259 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:36.786259 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:36.786259 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:36.786259 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:36.786259 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:36.786259 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 12:01:36.786259 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:36.786259 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 208/418] 10.222.0.98 () {56 vars in 1114 bytes} [Thu Aug 18 12:01:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:36.793225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22984c7-134a-49aa-bda4-0bcfea0ff20c None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:36.912502 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22984c7-134a-49aa-bda4-0bcfea0ff20c None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:36.942453 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:36.968422 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3adf0d-c5c6-4e1d-b44b-8dcfe9376200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:36.969090 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3adf0d-c5c6-4e1d-b44b-8dcfe9376200 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:36.969479 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3adf0d-c5c6-4e1d-b44b-8dcfe9376200 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:36.969833 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3adf0d-c5c6-4e1d-b44b-8dcfe9376200 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:36.970573 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3adf0d-c5c6-4e1d-b44b-8dcfe9376200 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:36.970947 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3adf0d-c5c6-4e1d-b44b-8dcfe9376200 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:36.973338 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3adf0d-c5c6-4e1d-b44b-8dcfe9376200 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:36.973780 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ffd866-7742-46a5-adba-0edbf8a0ffc9 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:36.974070 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ffd866-7742-46a5-adba-0edbf8a0ffc9 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:36.974351 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ffd866-7742-46a5-adba-0edbf8a0ffc9 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:36.977215 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 209/419] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:36.981914 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70757ff2-2eb3-489f-a0ee-b8daf46f8c32 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:36.982313 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70757ff2-2eb3-489f-a0ee-b8daf46f8c32 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:36.982667 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70757ff2-2eb3-489f-a0ee-b8daf46f8c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:36.988407 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 211/420] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:01:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:36.993578 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3adf0d-c5c6-4e1d-b44b-8dcfe9376200 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:37.019591 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-70757ff2-2eb3-489f-a0ee-b8daf46f8c32 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:37.020804 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-70757ff2-2eb3-489f-a0ee-b8daf46f8c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:37.144908 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 210/421] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:37.208397 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3adf0d-c5c6-4e1d-b44b-8dcfe9376200 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:37.265507 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae01dbb-31a8-40c9-929c-ec3777509965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:37.266149 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae01dbb-31a8-40c9-929c-ec3777509965 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:37.266433 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae01dbb-31a8-40c9-929c-ec3777509965 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:37.266712 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae01dbb-31a8-40c9-929c-ec3777509965 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:37.340652 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:37.368942 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae01dbb-31a8-40c9-929c-ec3777509965 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:37.370628 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae01dbb-31a8-40c9-929c-ec3777509965 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:37.509631 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 212/422] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:01:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:01:37.515911 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70757ff2-2eb3-489f-a0ee-b8daf46f8c32 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:37.590355 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:37.652478 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70757ff2-2eb3-489f-a0ee-b8daf46f8c32 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:37.718505 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd65ba-3c49-438f-80c1-60057117f60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:37.719149 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd65ba-3c49-438f-80c1-60057117f60d None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:37.719451 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd65ba-3c49-438f-80c1-60057117f60d None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:37.719731 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd65ba-3c49-438f-80c1-60057117f60d None admin] PATH_INFO: `/v3/projects/dede24cb1b014e9796b44d6cdac3ed69` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:37.720888 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bd65ba-3c49-438f-80c1-60057117f60d None admin] RBAC: Authorizing `identity:get_project(project_id=dede24cb1b014e9796b44d6cdac3ed69)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:37.722333 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bd65ba-3c49-438f-80c1-60057117f60d None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:37.723993 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 211/423] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:01:37 2022] GET /identity/v3/projects/dede24cb1b014e9796b44d6cdac3ed69 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:37.731453 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae01dbb-31a8-40c9-929c-ec3777509965 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:37.798646 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd65ba-3c49-438f-80c1-60057117f60d None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:37.903799 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:37.923165 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae01dbb-31a8-40c9-929c-ec3777509965 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:37.984218 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd7d7a-5641-4301-9229-a7ffe3636edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:37.984857 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd7d7a-5641-4301-9229-a7ffe3636edf None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:37.985139 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd7d7a-5641-4301-9229-a7ffe3636edf None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:37.985411 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd7d7a-5641-4301-9229-a7ffe3636edf None admin] PATH_INFO: `/v3/groups/9154e4b2dc4c4aa68e28f6e3d70f783a` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:37.986646 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd7d7a-5641-4301-9229-a7ffe3636edf None admin] RBAC: Authorizing `identity:get_group(group_id=9154e4b2dc4c4aa68e28f6e3d70f783a)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:37.988205 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd7d7a-5641-4301-9229-a7ffe3636edf None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:37.990086 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 213/424] 10.222.0.98 () {56 vars in 1180 bytes} [Thu Aug 18 12:01:37 2022] GET /identity/v3/groups/9154e4b2dc4c4aa68e28f6e3d70f783a => generated 226 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:01:37.995452 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0b635-4546-4da6-a954-87fbf0fc7a08 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:37.995452 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0b635-4546-4da6-a954-87fbf0fc7a08 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:37.995452 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0b635-4546-4da6-a954-87fbf0fc7a08 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:37.997193 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 214/425] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:37.999184 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a89928-4d27-4b6e-8ed6-9feb7d2f2c53 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:37.999506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a89928-4d27-4b6e-8ed6-9feb7d2f2c53 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:37.999781 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a89928-4d27-4b6e-8ed6-9feb7d2f2c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:38.020013 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-c0a89928-4d27-4b6e-8ed6-9feb7d2f2c53 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:38.021396 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-c0a89928-4d27-4b6e-8ed6-9feb7d2f2c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:38.028757 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd65ba-3c49-438f-80c1-60057117f60d None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:38.054841 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:38.067848 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b4f242-333a-4653-b21e-f1b13d3f9a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:38.068736 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4f242-333a-4653-b21e-f1b13d3f9a6b None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:38.069168 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4f242-333a-4653-b21e-f1b13d3f9a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:38.073681 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4f242-333a-4653-b21e-f1b13d3f9a6b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:38.082470 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b4f242-333a-4653-b21e-f1b13d3f9a6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:38.087954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-59b4f242-333a-4653-b21e-f1b13d3f9a6b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 12:01:38.087954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:38.087954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:38.087954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:38.087954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:38.087954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:38.087954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:38.087954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:38.087954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:38.087954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:38.087954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:38.087954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:38.087954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:38.087954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:38.087954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:38.087954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:38.089119 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:38.089119 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:38.089119 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:38.089119 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:38.089119 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:38.089119 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:38.089119 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:38.089119 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:38.089119 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:38.089119 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:38.089119 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:38.089119 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:38.089119 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:38.089119 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:38.089119 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:38.089119 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:38.090114 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:38.090114 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:38.090114 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:38.090114 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:38.090114 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:38.090114 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:38.090114 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:38.090114 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:38.090114 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:38.090114 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:38.090114 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 12:01:38.090114 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:38.092621 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 212/426] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:01:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:38.095460 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b4f242-333a-4653-b21e-f1b13d3f9a6b None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:38.103765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 215/427] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 12:01:38.104723 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a89928-4d27-4b6e-8ed6-9feb7d2f2c53 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:38.215089 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:38.251892 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a89928-4d27-4b6e-8ed6-9feb7d2f2c53 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:38.315234 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b4f242-333a-4653-b21e-f1b13d3f9a6b None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:38.325149 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57fff77-56cd-4683-bb62-74533cdba29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:38.326127 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57fff77-56cd-4683-bb62-74533cdba29a None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:38.326968 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57fff77-56cd-4683-bb62-74533cdba29a None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:38.327452 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57fff77-56cd-4683-bb62-74533cdba29a None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:38.328624 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57fff77-56cd-4683-bb62-74533cdba29a None admin] RBAC: Adding query filter params (name=member) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:38.329362 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57fff77-56cd-4683-bb62-74533cdba29a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:38.331104 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57fff77-56cd-4683-bb62-74533cdba29a None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:38.343105 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 216/428] 10.222.0.98 () {56 vars in 1101 bytes} [Thu Aug 18 12:01:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:38.344763 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eac1e3-c49a-4ba7-8004-3a5cb6bd518c None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:38.345214 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eac1e3-c49a-4ba7-8004-3a5cb6bd518c None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:38.345662 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eac1e3-c49a-4ba7-8004-3a5cb6bd518c None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:38.347291 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 217/429] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:38.348872 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eac1e3-c49a-4ba7-8004-3a5cb6bd518c None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:38.359853 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04397370-d490-4d81-9db5-fb8373cddad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:38.360596 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04397370-d490-4d81-9db5-fb8373cddad1 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:38.360950 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04397370-d490-4d81-9db5-fb8373cddad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:38.361344 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04397370-d490-4d81-9db5-fb8373cddad1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:38.362634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04397370-d490-4d81-9db5-fb8373cddad1 None admin] RBAC: Adding query filter params (group.id=9154e4b2dc4c4aa68e28f6e3d70f783a, role.id=3a4b7e53b8644e508ac440f71138be16, scope.project.id=dede24cb1b014e9796b44d6cdac3ed69) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:38.363036 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04397370-d490-4d81-9db5-fb8373cddad1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:38.364978 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04397370-d490-4d81-9db5-fb8373cddad1 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:38.378504 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 213/430] 10.222.0.98 () {56 vars in 1376 bytes} [Thu Aug 18 12:01:38 2022] GET /identity/v3/role_assignments?group.id=9154e4b2dc4c4aa68e28f6e3d70f783a&scope.project.id=dede24cb1b014e9796b44d6cdac3ed69&role.id=3a4b7e53b8644e508ac440f71138be16 => generated 611 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:38.380038 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5127db-9ea8-46aa-bbf4-4783b157de6f None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:38.380401 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5127db-9ea8-46aa-bbf4-4783b157de6f None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:38.380754 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5127db-9ea8-46aa-bbf4-4783b157de6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:38.403476 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-ee5127db-9ea8-46aa-bbf4-4783b157de6f None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:38.405425 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-ee5127db-9ea8-46aa-bbf4-4783b157de6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:38.466333 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:38.501926 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 214/431] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:38.503148 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5127db-9ea8-46aa-bbf4-4783b157de6f None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:38.522258 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eac1e3-c49a-4ba7-8004-3a5cb6bd518c None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:38.578787 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a426ed-ed84-4852-ae57-7e8f270c22db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:38.579447 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a426ed-ed84-4852-ae57-7e8f270c22db None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:38.579826 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a426ed-ed84-4852-ae57-7e8f270c22db None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:38.580227 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a426ed-ed84-4852-ae57-7e8f270c22db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:38.592808 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:38.622218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5127db-9ea8-46aa-bbf4-4783b157de6f None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:38.686925 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a426ed-ed84-4852-ae57-7e8f270c22db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:38.688841 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a426ed-ed84-4852-ae57-7e8f270c22db None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:38.692394 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dbfa00-6512-4dc5-b150-9a6ecdbaf713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:38.694524 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbfa00-6512-4dc5-b150-9a6ecdbaf713 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:38.700224 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbfa00-6512-4dc5-b150-9a6ecdbaf713 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:38.701401 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbfa00-6512-4dc5-b150-9a6ecdbaf713 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:38.719354 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dbfa00-6512-4dc5-b150-9a6ecdbaf713 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:38.724824 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-c4dbfa00-6512-4dc5-b150-9a6ecdbaf713 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 18 12:01:38.724824 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:38.724824 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:38.724824 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:38.724824 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:38.724824 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:38.724824 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:38.724824 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:38.724824 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:38.724824 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:38.724824 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:38.724824 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:38.724824 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:38.724824 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:38.724824 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:38.724824 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:38.725758 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:38.725758 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:38.725758 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:38.725758 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:38.725758 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:38.725758 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:38.725758 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:38.725758 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:38.725758 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:38.725758 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:38.725758 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:38.725758 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:38.725758 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:38.725758 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:38.725758 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:38.725758 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:38.726570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:38.726570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:38.726570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:38.726570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:38.726570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:38.726570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:38.726570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:38.726570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:38.726570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:38.726570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:38.726570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 18 12:01:38.726570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:38.726570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 215/432] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:01:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:38.732241 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dbfa00-6512-4dc5-b150-9a6ecdbaf713 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:38.781146 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:38.862415 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 218/433] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:01:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:01:38.867763 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a426ed-ed84-4852-ae57-7e8f270c22db None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:38.885251 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dbfa00-6512-4dc5-b150-9a6ecdbaf713 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:38.909314 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:38.929608 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f988d-1ef6-4ab0-89db-a02a5caa8434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:38.930606 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f988d-1ef6-4ab0-89db-a02a5caa8434 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:38.931043 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f988d-1ef6-4ab0-89db-a02a5caa8434 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:38.931319 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f988d-1ef6-4ab0-89db-a02a5caa8434 None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:38.932235 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f988d-1ef6-4ab0-89db-a02a5caa8434 None admin] RBAC: Adding query filter params (name=member) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:38.932543 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f988d-1ef6-4ab0-89db-a02a5caa8434 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:38.934061 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f988d-1ef6-4ab0-89db-a02a5caa8434 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:38.949000 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 216/434] 10.222.0.98 () {56 vars in 1101 bytes} [Thu Aug 18 12:01:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:01:38.949690 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f988d-1ef6-4ab0-89db-a02a5caa8434 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:38.982734 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a426ed-ed84-4852-ae57-7e8f270c22db None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:39.020772 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d0b14-9881-4f8f-8035-5701c77195a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:39.021403 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d0b14-9881-4f8f-8035-5701c77195a5 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:39.021688 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d0b14-9881-4f8f-8035-5701c77195a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:39.022205 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d0b14-9881-4f8f-8035-5701c77195a5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:39.032914 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d0b14-9881-4f8f-8035-5701c77195a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:39.037707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-f09d0b14-9881-4f8f-8035-5701c77195a5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 12:01:39.037707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:39.037707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:39.037707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:39.037707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:39.037707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:39.037707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:39.037707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:39.037707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:39.037707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:39.037707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:39.037707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:39.037707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:39.037707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:39.037707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:39.037707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:39.038642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:39.038642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:39.038642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:39.038642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:39.038642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:39.038642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:39.038642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:39.038642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:39.038642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:39.038642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:39.038642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:39.038642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:39.038642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:39.038642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:39.038642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:39.038642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:39.039496 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:39.039496 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:39.039496 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:39.039496 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:39.039496 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:39.039496 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:39.039496 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:39.039496 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:39.039496 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:39.039496 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:39.039496 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 12:01:39.039496 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:39.039496 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 219/435] 10.222.0.98 () {56 vars in 1114 bytes} [Thu Aug 18 12:01:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:39.040691 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d0b14-9881-4f8f-8035-5701c77195a5 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:39.064420 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:39.068696 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f988d-1ef6-4ab0-89db-a02a5caa8434 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:39.103138 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e3f44-d8ee-4d91-802c-c47b94c603e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:39.103823 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e3f44-d8ee-4d91-802c-c47b94c603e8 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:39.104112 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e3f44-d8ee-4d91-802c-c47b94c603e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:39.104482 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e3f44-d8ee-4d91-802c-c47b94c603e8 None admin] PATH_INFO: `/v3/projects/dede24cb1b014e9796b44d6cdac3ed69` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:39.105658 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e3f44-d8ee-4d91-802c-c47b94c603e8 None admin] RBAC: Authorizing `identity:get_project(project_id=dede24cb1b014e9796b44d6cdac3ed69)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:39.107103 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e3f44-d8ee-4d91-802c-c47b94c603e8 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:39.108794 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 217/436] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:01:38 2022] GET /identity/v3/projects/dede24cb1b014e9796b44d6cdac3ed69 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:39.109916 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1510b-d0bf-457e-a7e5-fcf906fb41ce None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:39.110203 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1510b-d0bf-457e-a7e5-fcf906fb41ce None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:39.110484 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1510b-d0bf-457e-a7e5-fcf906fb41ce None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:39.111696 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 218/437] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:39.112529 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1510b-d0bf-457e-a7e5-fcf906fb41ce None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:39.141247 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:39.145002 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d0b14-9881-4f8f-8035-5701c77195a5 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:39.179657 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4adf9d-0de2-4692-adc4-a5a35fb1186e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:39.180279 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4adf9d-0de2-4692-adc4-a5a35fb1186e None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:39.180568 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4adf9d-0de2-4692-adc4-a5a35fb1186e None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:39.180851 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4adf9d-0de2-4692-adc4-a5a35fb1186e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:39.214295 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1510b-d0bf-457e-a7e5-fcf906fb41ce None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:39.247994 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d11b51-15c9-4264-9dbd-2771d777229d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:39.248618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d11b51-15c9-4264-9dbd-2771d777229d None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:39.249397 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4adf9d-0de2-4692-adc4-a5a35fb1186e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:39.249839 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d11b51-15c9-4264-9dbd-2771d777229d None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:39.249839 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d11b51-15c9-4264-9dbd-2771d777229d None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:39.250371 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d11b51-15c9-4264-9dbd-2771d777229d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:39.250681 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d11b51-15c9-4264-9dbd-2771d777229d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:39.250867 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4adf9d-0de2-4692-adc4-a5a35fb1186e None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:39.252058 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d11b51-15c9-4264-9dbd-2771d777229d None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:39.260561 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 219/438] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:01:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:39.261359 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d11b51-15c9-4264-9dbd-2771d777229d None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:39.288710 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:39.345180 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:39.349577 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 220/439] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:01:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:01:39.350899 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efead774-7d04-4f2f-b762-cc1f8f2f8099 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:39.351240 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efead774-7d04-4f2f-b762-cc1f8f2f8099 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:39.351566 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efead774-7d04-4f2f-b762-cc1f8f2f8099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:39.361031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d11b51-15c9-4264-9dbd-2771d777229d None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:39.369635 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-efead774-7d04-4f2f-b762-cc1f8f2f8099 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:39.370778 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-efead774-7d04-4f2f-b762-cc1f8f2f8099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:39.394613 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7341c-06f7-4cc2-8f85-48d770248378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:39.395246 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7341c-06f7-4cc2-8f85-48d770248378 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:39.395568 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7341c-06f7-4cc2-8f85-48d770248378 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:39.395838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7341c-06f7-4cc2-8f85-48d770248378 None admin] PATH_INFO: `/v3/groups/9154e4b2dc4c4aa68e28f6e3d70f783a` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:39.396963 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d7341c-06f7-4cc2-8f85-48d770248378 None admin] RBAC: Authorizing `identity:get_group(group_id=9154e4b2dc4c4aa68e28f6e3d70f783a)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:39.398660 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d7341c-06f7-4cc2-8f85-48d770248378 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:39.400255 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 220/440] 10.222.0.98 () {56 vars in 1180 bytes} [Thu Aug 18 12:01:39 2022] GET /identity/v3/groups/9154e4b2dc4c4aa68e28f6e3d70f783a => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:39.401016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7341c-06f7-4cc2-8f85-48d770248378 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:39.440889 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 221/441] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:39.441743 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efead774-7d04-4f2f-b762-cc1f8f2f8099 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:39.504436 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:39.512002 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7341c-06f7-4cc2-8f85-48d770248378 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:39.548746 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efead774-7d04-4f2f-b762-cc1f8f2f8099 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:39.549846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be58b681-0616-4f36-877c-c7d1bde17aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:39.550459 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58b681-0616-4f36-877c-c7d1bde17aec None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:39.550746 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58b681-0616-4f36-877c-c7d1bde17aec None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:39.551778 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58b681-0616-4f36-877c-c7d1bde17aec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:39.577234 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:39.585628 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd4e61a-9825-49b8-ab5f-8e067f4ce342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:39.586380 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd4e61a-9825-49b8-ab5f-8e067f4ce342 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:39.586715 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd4e61a-9825-49b8-ab5f-8e067f4ce342 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:39.586968 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd4e61a-9825-49b8-ab5f-8e067f4ce342 None admin] PATH_INFO: `/v3/projects/9d8b38972ba64f509734ed17f30c88fd` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:39.588122 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd4e61a-9825-49b8-ab5f-8e067f4ce342 None admin] RBAC: Authorizing `identity:get_project(project_id=9d8b38972ba64f509734ed17f30c88fd)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:39.593213 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd4e61a-9825-49b8-ab5f-8e067f4ce342 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:39.593213 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 222/442] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:01:39 2022] GET /identity/v3/projects/9d8b38972ba64f509734ed17f30c88fd => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:39.593213 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd4e61a-9825-49b8-ab5f-8e067f4ce342 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:39.628992 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be58b681-0616-4f36-877c-c7d1bde17aec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:39.630367 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be58b681-0616-4f36-877c-c7d1bde17aec None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:39.703844 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd4e61a-9825-49b8-ab5f-8e067f4ce342 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:39.731470 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:39.737819 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 221/443] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:01:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:01:39.738613 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be58b681-0616-4f36-877c-c7d1bde17aec None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:39.744198 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f5a629-d143-4c1c-b675-b8ef39e130c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:39.744818 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5a629-d143-4c1c-b675-b8ef39e130c3 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:39.745102 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5a629-d143-4c1c-b675-b8ef39e130c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:39.745381 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5a629-d143-4c1c-b675-b8ef39e130c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:39.746114 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f5a629-d143-4c1c-b675-b8ef39e130c3 None admin] RBAC: Adding query filter params (group.id=9154e4b2dc4c4aa68e28f6e3d70f783a, role.id=c81e28908b9b49a38d33978df4d11716, scope.project.id=dede24cb1b014e9796b44d6cdac3ed69) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:39.746430 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f5a629-d143-4c1c-b675-b8ef39e130c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:39.747893 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f5a629-d143-4c1c-b675-b8ef39e130c3 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:39.753949 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 223/444] 10.222.0.98 () {56 vars in 1376 bytes} [Thu Aug 18 12:01:39 2022] GET /identity/v3/role_assignments?group.id=9154e4b2dc4c4aa68e28f6e3d70f783a&scope.project.id=dede24cb1b014e9796b44d6cdac3ed69&role.id=c81e28908b9b49a38d33978df4d11716 => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:39.754736 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f5a629-d143-4c1c-b675-b8ef39e130c3 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:39.838265 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:39.859213 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be58b681-0616-4f36-877c-c7d1bde17aec None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:39.886105 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f5a629-d143-4c1c-b675-b8ef39e130c3 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:39.899118 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152fef0f-85a6-4917-881d-17a13d379de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:39.899778 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152fef0f-85a6-4917-881d-17a13d379de4 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:39.900061 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152fef0f-85a6-4917-881d-17a13d379de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:39.900341 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152fef0f-85a6-4917-881d-17a13d379de4 None admin] PATH_INFO: `/v3/groups/9154e4b2dc4c4aa68e28f6e3d70f783a` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:39.901529 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152fef0f-85a6-4917-881d-17a13d379de4 None admin] RBAC: Authorizing `identity:get_group(group_id=9154e4b2dc4c4aa68e28f6e3d70f783a)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:39.902978 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152fef0f-85a6-4917-881d-17a13d379de4 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:39.904591 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 222/445] 10.222.0.98 () {56 vars in 1180 bytes} [Thu Aug 18 12:01:39 2022] GET /identity/v3/groups/9154e4b2dc4c4aa68e28f6e3d70f783a => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:39.905931 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152fef0f-85a6-4917-881d-17a13d379de4 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:39.930229 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827f181b-83b1-4910-82e7-959a3d17ab92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:39.931486 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f181b-83b1-4910-82e7-959a3d17ab92 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:39.931486 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f181b-83b1-4910-82e7-959a3d17ab92 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:39.931486 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f181b-83b1-4910-82e7-959a3d17ab92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:39.944340 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827f181b-83b1-4910-82e7-959a3d17ab92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:39.950013 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-827f181b-83b1-4910-82e7-959a3d17ab92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:01:39.950013 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:39.950013 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:39.950013 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:39.950013 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:39.950013 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:39.950013 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:39.950013 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:39.950013 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:39.950013 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:39.950013 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:39.950013 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:39.950013 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:39.950013 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:39.950013 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:39.950013 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:39.950919 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:39.950919 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:39.950919 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:39.950919 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:39.950919 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:39.950919 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:39.950919 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:39.950919 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:39.950919 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:39.950919 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:39.950919 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:39.950919 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:39.950919 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:39.950919 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:39.950919 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:39.950919 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:39.951748 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:39.951748 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:39.951748 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:39.951748 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:39.951748 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:39.951748 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:39.951748 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:39.951748 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:39.951748 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:39.951748 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:39.951748 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:01:39.951748 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:39.951748 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 224/446] 10.222.0.98 () {56 vars in 1096 bytes} [Thu Aug 18 12:01:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:39.953443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827f181b-83b1-4910-82e7-959a3d17ab92 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:39.983923 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:40.033115 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152fef0f-85a6-4917-881d-17a13d379de4 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:40.075720 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f834b313-9ede-4cb2-b2f2-5e5a18a65da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:40.076633 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834b313-9ede-4cb2-b2f2-5e5a18a65da3 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:40.077511 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834b313-9ede-4cb2-b2f2-5e5a18a65da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:40.077779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834b313-9ede-4cb2-b2f2-5e5a18a65da3 None admin] PATH_INFO: `/v3/projects/9d8b38972ba64f509734ed17f30c88fd` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:40.078801 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827f181b-83b1-4910-82e7-959a3d17ab92 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:40.079344 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f834b313-9ede-4cb2-b2f2-5e5a18a65da3 None admin] RBAC: Authorizing `identity:get_project(project_id=9d8b38972ba64f509734ed17f30c88fd)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:40.081008 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f834b313-9ede-4cb2-b2f2-5e5a18a65da3 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:40.083647 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 223/447] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:01:39 2022] GET /identity/v3/projects/9d8b38972ba64f509734ed17f30c88fd => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:40.084261 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f834b313-9ede-4cb2-b2f2-5e5a18a65da3 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:40.101122 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:40.116823 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c711a-07a1-42e0-9153-03151b35cae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:40.117451 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c711a-07a1-42e0-9153-03151b35cae7 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:40.117738 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c711a-07a1-42e0-9153-03151b35cae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:40.118021 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c711a-07a1-42e0-9153-03151b35cae7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:40.119029 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7c711a-07a1-42e0-9153-03151b35cae7 None admin] RBAC: Adding query filter params (group.id=9154e4b2dc4c4aa68e28f6e3d70f783a, role.id=c81e28908b9b49a38d33978df4d11716, scope.project.id=9d8b38972ba64f509734ed17f30c88fd) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:40.119174 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7c711a-07a1-42e0-9153-03151b35cae7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:40.120630 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7c711a-07a1-42e0-9153-03151b35cae7 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:40.127346 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 225/448] 10.222.0.98 () {56 vars in 1376 bytes} [Thu Aug 18 12:01:39 2022] GET /identity/v3/role_assignments?group.id=9154e4b2dc4c4aa68e28f6e3d70f783a&scope.project.id=9d8b38972ba64f509734ed17f30c88fd&role.id=c81e28908b9b49a38d33978df4d11716 => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:40.128160 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c711a-07a1-42e0-9153-03151b35cae7 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:40.214537 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f834b313-9ede-4cb2-b2f2-5e5a18a65da3 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:40.258876 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:40.265005 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b9bbf-19ec-4fd5-ae26-9353e258df36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:40.265636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b9bbf-19ec-4fd5-ae26-9353e258df36 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:40.265916 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b9bbf-19ec-4fd5-ae26-9353e258df36 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:40.266361 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b9bbf-19ec-4fd5-ae26-9353e258df36 None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:40.267124 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8b9bbf-19ec-4fd5-ae26-9353e258df36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:40.268613 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8b9bbf-19ec-4fd5-ae26-9353e258df36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:40.268965 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8b9bbf-19ec-4fd5-ae26-9353e258df36 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:40.278859 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 224/449] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:01:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:40.284490 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b9bbf-19ec-4fd5-ae26-9353e258df36 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:40.296698 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c711a-07a1-42e0-9153-03151b35cae7 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:40.347767 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe1ff9-6ca2-4a52-aa4d-caca57d718b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:40.348777 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe1ff9-6ca2-4a52-aa4d-caca57d718b8 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:40.349074 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe1ff9-6ca2-4a52-aa4d-caca57d718b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:40.349421 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe1ff9-6ca2-4a52-aa4d-caca57d718b8 None admin] PATH_INFO: `/v3/groups/9154e4b2dc4c4aa68e28f6e3d70f783a` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:40.351167 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffe1ff9-6ca2-4a52-aa4d-caca57d718b8 None admin] RBAC: Authorizing `identity:get_group(group_id=9154e4b2dc4c4aa68e28f6e3d70f783a)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:40.354465 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffe1ff9-6ca2-4a52-aa4d-caca57d718b8 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:40.358301 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 226/450] 10.222.0.98 () {56 vars in 1180 bytes} [Thu Aug 18 12:01:40 2022] GET /identity/v3/groups/9154e4b2dc4c4aa68e28f6e3d70f783a => generated 226 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:40.364489 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe1ff9-6ca2-4a52-aa4d-caca57d718b8 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:40.426234 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b9bbf-19ec-4fd5-ae26-9353e258df36 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:40.435222 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:40.465234 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c499b4-997f-4063-99d9-fad31bd27550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:40.465764 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c499b4-997f-4063-99d9-fad31bd27550 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:40.466045 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c499b4-997f-4063-99d9-fad31bd27550 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:40.466317 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c499b4-997f-4063-99d9-fad31bd27550 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:40.503576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe1ff9-6ca2-4a52-aa4d-caca57d718b8 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:40.543989 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:40.554289 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c499b4-997f-4063-99d9-fad31bd27550 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:40.555784 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c499b4-997f-4063-99d9-fad31bd27550 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:40.563007 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c23944-c76e-44fa-ac82-485bbd5a4c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:40.563728 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c23944-c76e-44fa-ac82-485bbd5a4c5c None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:40.564014 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c23944-c76e-44fa-ac82-485bbd5a4c5c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:40.564293 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c23944-c76e-44fa-ac82-485bbd5a4c5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:40.565104 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c23944-c76e-44fa-ac82-485bbd5a4c5c None admin] RBAC: Adding query filter params (group.id=9154e4b2dc4c4aa68e28f6e3d70f783a, role.id=3a4b7e53b8644e508ac440f71138be16, scope.project.id=9d8b38972ba64f509734ed17f30c88fd) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:40.565423 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c23944-c76e-44fa-ac82-485bbd5a4c5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:40.566874 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c23944-c76e-44fa-ac82-485bbd5a4c5c None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:40.573859 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 227/451] 10.222.0.98 () {56 vars in 1376 bytes} [Thu Aug 18 12:01:40 2022] GET /identity/v3/role_assignments?group.id=9154e4b2dc4c4aa68e28f6e3d70f783a&scope.project.id=9d8b38972ba64f509734ed17f30c88fd&role.id=3a4b7e53b8644e508ac440f71138be16 => generated 611 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:01:40.676463 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 225/452] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:01:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 18 12:01:40.682808 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c23944-c76e-44fa-ac82-485bbd5a4c5c None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:40.813201 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c23944-c76e-44fa-ac82-485bbd5a4c5c None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:40.829658 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:40.847529 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd145e-54a5-493f-adb8-55f7d199a871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:40.848174 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd145e-54a5-493f-adb8-55f7d199a871 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:40.849585 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd145e-54a5-493f-adb8-55f7d199a871 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:40.849585 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd145e-54a5-493f-adb8-55f7d199a871 None admin] PATH_INFO: `/v3/projects/c192a2eba4e24841893715c8478b559c` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:40.849877 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd145e-54a5-493f-adb8-55f7d199a871 None admin] RBAC: Authorizing `identity:get_project(project_id=c192a2eba4e24841893715c8478b559c)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:40.851297 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd145e-54a5-493f-adb8-55f7d199a871 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:40.853034 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 228/453] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:01:40 2022] GET /identity/v3/projects/c192a2eba4e24841893715c8478b559c => generated 344 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:40.859636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c499b4-997f-4063-99d9-fad31bd27550 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:40.961073 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:40.968075 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c499b4-997f-4063-99d9-fad31bd27550 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:41.001847 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f37402-1bba-4e47-93ed-728becc5338d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:41.002457 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f37402-1bba-4e47-93ed-728becc5338d None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:41.002736 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f37402-1bba-4e47-93ed-728becc5338d None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:41.003009 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f37402-1bba-4e47-93ed-728becc5338d None admin] PATH_INFO: `/v3/groups/57ddb3b5c42d43ed838f1af808c71e69` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:41.004191 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f37402-1bba-4e47-93ed-728becc5338d None admin] RBAC: Authorizing `identity:get_group(group_id=57ddb3b5c42d43ed838f1af808c71e69)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:41.005618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f37402-1bba-4e47-93ed-728becc5338d None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:41.007649 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 226/454] 10.222.0.98 () {56 vars in 1180 bytes} [Thu Aug 18 12:01:40 2022] GET /identity/v3/groups/57ddb3b5c42d43ed838f1af808c71e69 => generated 229 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:41.013526 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd145e-54a5-493f-adb8-55f7d199a871 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:41.111481 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd145e-54a5-493f-adb8-55f7d199a871 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:41.143132 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f0d7cb-4b27-4409-a50b-f882b67579e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:41.143756 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f0d7cb-4b27-4409-a50b-f882b67579e8 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:41.144042 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f0d7cb-4b27-4409-a50b-f882b67579e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:41.144316 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f0d7cb-4b27-4409-a50b-f882b67579e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:41.145029 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f0d7cb-4b27-4409-a50b-f882b67579e8 None admin] RBAC: Adding query filter params (group.id=57ddb3b5c42d43ed838f1af808c71e69, role.id=c53a0e13b58642f291c856efe1f0b87d, scope.project.id=c192a2eba4e24841893715c8478b559c) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:41.145338 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f0d7cb-4b27-4409-a50b-f882b67579e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:41.146750 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f0d7cb-4b27-4409-a50b-f882b67579e8 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:41.152573 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 229/455] 10.222.0.98 () {56 vars in 1376 bytes} [Thu Aug 18 12:01:41 2022] GET /identity/v3/role_assignments?group.id=57ddb3b5c42d43ed838f1af808c71e69&scope.project.id=c192a2eba4e24841893715c8478b559c&role.id=c53a0e13b58642f291c856efe1f0b87d => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:43.496618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e1deb-0393-44ff-bbd5-415206e100cc None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:43.497169 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e1deb-0393-44ff-bbd5-415206e100cc None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:43.497169 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e1deb-0393-44ff-bbd5-415206e100cc None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:43.498798 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 227/456] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:43.513549 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67de94b-415a-4559-84bc-8a6a6c7642d9 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:43.514046 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67de94b-415a-4559-84bc-8a6a6c7642d9 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:43.514151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67de94b-415a-4559-84bc-8a6a6c7642d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:43.536159 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-c67de94b-415a-4559-84bc-8a6a6c7642d9 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:43.537359 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-c67de94b-415a-4559-84bc-8a6a6c7642d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:43.599647 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:43.628127 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 230/457] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:43.852463 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25225b63-6253-454b-b627-a9c61e4a759f None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:43.852463 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25225b63-6253-454b-b627-a9c61e4a759f None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:43.852463 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25225b63-6253-454b-b627-a9c61e4a759f None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:43.853210 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 228/458] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:43.859377 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65dd0e-27d4-4567-acf0-f68e6f9c5cd9 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:43.859971 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65dd0e-27d4-4567-acf0-f68e6f9c5cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:43.860451 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65dd0e-27d4-4567-acf0-f68e6f9c5cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:43.877288 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664193e7-97b0-46ee-851d-98481e08a50b None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:43.877608 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664193e7-97b0-46ee-851d-98481e08a50b None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:43.877857 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664193e7-97b0-46ee-851d-98481e08a50b None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:43.879449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 229/459] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:43.879962 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-1b65dd0e-27d4-4567-acf0-f68e6f9c5cd9 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:43.881359 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-1b65dd0e-27d4-4567-acf0-f68e6f9c5cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:43.885238 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af69ef9-fa7d-4ae9-a95e-3213d8c0298f None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:43.885238 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af69ef9-fa7d-4ae9-a95e-3213d8c0298f None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:43.885238 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af69ef9-fa7d-4ae9-a95e-3213d8c0298f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:43.924494 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-4af69ef9-fa7d-4ae9-a95e-3213d8c0298f None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:43.925727 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-4af69ef9-fa7d-4ae9-a95e-3213d8c0298f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:43.955713 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 231/460] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:44.030027 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65dd0e-27d4-4567-acf0-f68e6f9c5cd9 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:44.053443 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 230/461] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:44.138147 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65dd0e-27d4-4567-acf0-f68e6f9c5cd9 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:44.149304 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:44.170280 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826e0e4-29ff-44db-b50e-336d62be6d6f None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:44.170561 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826e0e4-29ff-44db-b50e-336d62be6d6f None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:44.170893 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826e0e4-29ff-44db-b50e-336d62be6d6f None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:44.179589 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 231/462] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 12:01:44.179589 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b2159-131a-4af0-a179-6fee5ecbdffd None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:44.179589 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b2159-131a-4af0-a179-6fee5ecbdffd None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:44.179589 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b2159-131a-4af0-a179-6fee5ecbdffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:44.185769 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e5f07-8f64-485b-b9c4-e56e03d904a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:44.186418 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e5f07-8f64-485b-b9c4-e56e03d904a1 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:44.186728 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e5f07-8f64-485b-b9c4-e56e03d904a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:44.187002 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e5f07-8f64-485b-b9c4-e56e03d904a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:44.209281 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-dc7b2159-131a-4af0-a179-6fee5ecbdffd None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:44.210519 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-dc7b2159-131a-4af0-a179-6fee5ecbdffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:44.214017 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e5f07-8f64-485b-b9c4-e56e03d904a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:44.215563 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e5f07-8f64-485b-b9c4-e56e03d904a1 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:44.242088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-d07e5f07-8f64-485b-b9c4-e56e03d904a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:44.242088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:44.242088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:44.242088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:44.242088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:44.242088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:44.242088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:44.242088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:44.242088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:44.242088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:44.242088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:44.242088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:44.242088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:01:44.242088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:01:44.242088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:01:44.242088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:01:44.243082 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:44.243082 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:44.243082 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:44.243082 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:44.243082 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:44.243082 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:44.243082 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:44.243082 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:44.243082 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:44.243082 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:44.243082 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:44.243082 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:44.243082 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:44.243082 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:44.243082 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:44.243082 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:44.243082 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:01:44.243974 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:01:44.243974 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:44.243974 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:44.247513 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 232/463] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:01:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:01:44.250700 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e5f07-8f64-485b-b9c4-e56e03d904a1 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:44.320171 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 232/464] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 12:01:44.321022 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7b2159-131a-4af0-a179-6fee5ecbdffd None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:44.370046 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:44.384742 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e5f07-8f64-485b-b9c4-e56e03d904a1 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:44.423382 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7901d8f-a44a-4316-a63c-9e37b8cf27de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:44.424913 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7901d8f-a44a-4316-a63c-9e37b8cf27de None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:44.425409 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7901d8f-a44a-4316-a63c-9e37b8cf27de None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:44.425890 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7901d8f-a44a-4316-a63c-9e37b8cf27de None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:44.426767 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7901d8f-a44a-4316-a63c-9e37b8cf27de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:44.427408 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7901d8f-a44a-4316-a63c-9e37b8cf27de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:44.429841 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7901d8f-a44a-4316-a63c-9e37b8cf27de None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:44.446861 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 233/465] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:01:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:44.448035 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7901d8f-a44a-4316-a63c-9e37b8cf27de None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:44.470653 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7b2159-131a-4af0-a179-6fee5ecbdffd None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:44.476786 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:44.509256 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ad73c-bbab-474e-9c26-fffc96c5bc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:44.509958 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ad73c-bbab-474e-9c26-fffc96c5bc6f None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:44.510467 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ad73c-bbab-474e-9c26-fffc96c5bc6f None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:44.510923 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ad73c-bbab-474e-9c26-fffc96c5bc6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:44.522639 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ad73c-bbab-474e-9c26-fffc96c5bc6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:44.524608 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ad73c-bbab-474e-9c26-fffc96c5bc6f None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:44.541509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-b70ad73c-bbab-474e-9c26-fffc96c5bc6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:44.541509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:44.541509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:44.541509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:44.541509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:44.541509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:44.541509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:44.541509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:44.541509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:44.541509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:44.541509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:44.541509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:44.541509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:01:44.541509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:01:44.541509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:01:44.541509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:01:44.542449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:44.542449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:44.542449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:44.542449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:44.542449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:44.542449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:44.542449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:44.542449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:44.542449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:44.542449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:44.542449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:44.542449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:44.542449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:44.542449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:44.542449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:44.542449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:44.542449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:01:44.543339 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:01:44.543339 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:44.543339 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:44.545445 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 233/466] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:01:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:44.547023 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ad73c-bbab-474e-9c26-fffc96c5bc6f None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:44.578344 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7901d8f-a44a-4316-a63c-9e37b8cf27de None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:44.614154 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0cb17a-68d9-4eed-8e2b-aaac8a758fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:44.614791 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0cb17a-68d9-4eed-8e2b-aaac8a758fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:44.615078 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0cb17a-68d9-4eed-8e2b-aaac8a758fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:44.615356 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0cb17a-68d9-4eed-8e2b-aaac8a758fa3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:44.625957 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0cb17a-68d9-4eed-8e2b-aaac8a758fa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:44.627436 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0cb17a-68d9-4eed-8e2b-aaac8a758fa3 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:44.636853 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:44.643023 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-be0cb17a-68d9-4eed-8e2b-aaac8a758fa3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:44.643023 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:44.643023 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:44.643023 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:44.643023 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:44.643023 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:44.643023 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:44.643023 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:44.643023 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:44.643023 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:44.643023 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:44.643023 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:44.643023 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:01:44.643023 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:01:44.643023 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:01:44.643023 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:01:44.644038 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:44.644038 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:44.644038 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:44.644038 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:44.644038 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:44.644038 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:44.644038 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:44.644038 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:44.644038 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:44.644038 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:44.644038 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:44.644038 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:44.644038 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:44.644038 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:44.644038 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:44.644038 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:44.644038 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:01:44.644905 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:01:44.644905 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:44.644905 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:44.644905 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 234/467] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:01:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:44.645869 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0cb17a-68d9-4eed-8e2b-aaac8a758fa3 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:44.667637 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ad73c-bbab-474e-9c26-fffc96c5bc6f None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:44.708186 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ea704-f52f-4743-92a0-fda3def749d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:44.708804 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ea704-f52f-4743-92a0-fda3def749d1 None admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:44.709097 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ea704-f52f-4743-92a0-fda3def749d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:44.709374 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ea704-f52f-4743-92a0-fda3def749d1 None admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:44.710015 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ea704-f52f-4743-92a0-fda3def749d1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:44.711556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ea704-f52f-4743-92a0-fda3def749d1 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:44.732914 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:44.762516 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0cb17a-68d9-4eed-8e2b-aaac8a758fa3 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:44.798352 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5e8c0-1325-4950-9663-6cc903a5dee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:44.798965 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5e8c0-1325-4950-9663-6cc903a5dee5 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:44.799248 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5e8c0-1325-4950-9663-6cc903a5dee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:44.799564 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5e8c0-1325-4950-9663-6cc903a5dee5 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:44.800224 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed5e8c0-1325-4950-9663-6cc903a5dee5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:44.800529 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed5e8c0-1325-4950-9663-6cc903a5dee5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:44.801917 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed5e8c0-1325-4950-9663-6cc903a5dee5 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:44.819235 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 235/468] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:01:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:44.819235 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5e8c0-1325-4950-9663-6cc903a5dee5 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:44.820102 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 234/469] 10.222.0.98 () {60 vars in 1132 bytes} [Thu Aug 18 12:01:44 2022] POST /identity/v3/users => generated 246 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:01:44.820102 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ea704-f52f-4743-92a0-fda3def749d1 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:44.959439 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5e8c0-1325-4950-9663-6cc903a5dee5 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:44.963219 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ea704-f52f-4743-92a0-fda3def749d1 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:44.995072 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:44.999564 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:45.002390 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47704619-5d7b-40d3-a4d0-83f8f2b62552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:45.003036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47704619-5d7b-40d3-a4d0-83f8f2b62552 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:45.003359 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47704619-5d7b-40d3-a4d0-83f8f2b62552 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:45.003802 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47704619-5d7b-40d3-a4d0-83f8f2b62552 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:45.009733 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57392253-e165-4e23-80b1-9981895d99c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:45.010159 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57392253-e165-4e23-80b1-9981895d99c3 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:45.010442 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57392253-e165-4e23-80b1-9981895d99c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:45.011212 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57392253-e165-4e23-80b1-9981895d99c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:45.011930 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57392253-e165-4e23-80b1-9981895d99c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:45.012243 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57392253-e165-4e23-80b1-9981895d99c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:45.014062 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57392253-e165-4e23-80b1-9981895d99c3 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:45.016212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47704619-5d7b-40d3-a4d0-83f8f2b62552 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:45.017678 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47704619-5d7b-40d3-a4d0-83f8f2b62552 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:45.036058 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-47704619-5d7b-40d3-a4d0-83f8f2b62552 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:45.036058 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:45.036058 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:45.036058 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:45.036058 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:45.036058 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:45.036058 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:45.036058 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:45.036058 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:45.036058 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:45.036058 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:45.036058 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:45.036058 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:01:45.036058 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:01:45.036058 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:01:45.036058 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:01:45.038498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:45.038498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:45.038498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:45.038498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:45.038498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:45.038498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:45.038498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:45.038498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:45.038498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:45.038498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:45.038498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:45.038498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:45.038498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:45.038498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:45.038498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:45.038498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:45.038498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:01:45.039464 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:01:45.039464 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:45.039464 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:45.039632 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 235/470] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:01:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:45.039703 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 236/471] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:01:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:45.039767 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57392253-e165-4e23-80b1-9981895d99c3 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:45.046668 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47704619-5d7b-40d3-a4d0-83f8f2b62552 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:45.148681 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57392253-e165-4e23-80b1-9981895d99c3 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:45.160984 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47704619-5d7b-40d3-a4d0-83f8f2b62552 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:45.187505 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4030b5f-6b25-4d60-a334-4cf8b037ad81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:45.188114 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4030b5f-6b25-4d60-a334-4cf8b037ad81 None admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:45.188394 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4030b5f-6b25-4d60-a334-4cf8b037ad81 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:45.188669 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4030b5f-6b25-4d60-a334-4cf8b037ad81 None admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:45.189325 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4030b5f-6b25-4d60-a334-4cf8b037ad81 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:45.190762 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4030b5f-6b25-4d60-a334-4cf8b037ad81 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:45.195865 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac89117-9058-4b41-8f91-5ae3c6d64edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:45.196478 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac89117-9058-4b41-8f91-5ae3c6d64edd None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:45.196765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac89117-9058-4b41-8f91-5ae3c6d64edd None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:45.197044 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac89117-9058-4b41-8f91-5ae3c6d64edd None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:45.197697 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac89117-9058-4b41-8f91-5ae3c6d64edd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:45.198008 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac89117-9058-4b41-8f91-5ae3c6d64edd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:45.200201 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac89117-9058-4b41-8f91-5ae3c6d64edd None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:45.214983 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 237/472] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:01:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:45.214983 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac89117-9058-4b41-8f91-5ae3c6d64edd None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:45.262369 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:45.300332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 236/473] 10.222.0.98 () {60 vars in 1132 bytes} [Thu Aug 18 12:01:45 2022] POST /identity/v3/users => generated 248 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:01:45.302148 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3d753-7f51-44b6-b054-b6d4e398ac27 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:45.302440 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3d753-7f51-44b6-b054-b6d4e398ac27 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:45.302794 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3d753-7f51-44b6-b054-b6d4e398ac27 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:45.305963 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 237/474] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:45.305963 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b3d753-7f51-44b6-b054-b6d4e398ac27 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:45.330919 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac89117-9058-4b41-8f91-5ae3c6d64edd None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:45.372327 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:45.375590 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b466414-f326-4330-95c5-344cef07f3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:45.376524 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b466414-f326-4330-95c5-344cef07f3ea None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:45.377006 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b466414-f326-4330-95c5-344cef07f3ea None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:45.377477 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b466414-f326-4330-95c5-344cef07f3ea None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:45.378371 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b466414-f326-4330-95c5-344cef07f3ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:45.380084 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b466414-f326-4330-95c5-344cef07f3ea None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:45.438256 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b3d753-7f51-44b6-b054-b6d4e398ac27 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:45.474863 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a18a6-1ea2-4ee8-8740-206a2645a731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:45.475403 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a18a6-1ea2-4ee8-8740-206a2645a731 None admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:45.475721 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a18a6-1ea2-4ee8-8740-206a2645a731 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:45.476004 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a18a6-1ea2-4ee8-8740-206a2645a731 None admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:45.476699 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3a18a6-1ea2-4ee8-8740-206a2645a731 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:45.478150 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3a18a6-1ea2-4ee8-8740-206a2645a731 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:45.507384 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 238/475] 10.222.0.98 () {60 vars in 1132 bytes} [Thu Aug 18 12:01:45 2022] POST /identity/v3/users => generated 248 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:01:45.509098 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e0139f-67e9-4634-98b0-4d5d16e26e85 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:45.509607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e0139f-67e9-4634-98b0-4d5d16e26e85 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:45.509902 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e0139f-67e9-4634-98b0-4d5d16e26e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:45.532966 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-d8e0139f-67e9-4634-98b0-4d5d16e26e85 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:45.534210 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-d8e0139f-67e9-4634-98b0-4d5d16e26e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:45.571626 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 238/476] 10.222.0.98 () {60 vars in 1132 bytes} [Thu Aug 18 12:01:45 2022] POST /identity/v3/users => generated 249 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:01:45.637033 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 239/477] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:45.939867 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a18a6-1ea2-4ee8-8740-206a2645a731 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:46.127270 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a18a6-1ea2-4ee8-8740-206a2645a731 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:46.129693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:46.184724 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbba7b2-f999-4607-91b8-bad5edd95687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:46.185344 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbba7b2-f999-4607-91b8-bad5edd95687 None admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:46.185623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbba7b2-f999-4607-91b8-bad5edd95687 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:46.185893 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbba7b2-f999-4607-91b8-bad5edd95687 None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:46.187078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbba7b2-f999-4607-91b8-bad5edd95687 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:46.188505 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbba7b2-f999-4607-91b8-bad5edd95687 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:46.231602 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.sql.core [None req-9cbba7b2-f999-4607-91b8-bad5edd95687 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 18 12:01:46.231602 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 18 12:01:46.231602 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [parameters: {'id': '1b710c59b9884804af7af60d2001329f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 18 12:01:46.231602 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81863) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 18 12:01:46.248032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-9cbba7b2-f999-4607-91b8-bad5edd95687 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 18 12:01:46.248032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:46.248032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 18 12:01:46.248032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 18 12:01:46.248032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 18 12:01:46.248032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 18 12:01:46.248032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 18 12:01:46.248032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application result = self._query(query) Aug 18 12:01:46.248032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 18 12:01:46.248032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application conn.query(q) Aug 18 12:01:46.248032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 18 12:01:46.248032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 18 12:01:46.248032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 18 12:01:46.248032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application result.read() Aug 18 12:01:46.248032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 18 12:01:46.248937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 18 12:01:46.248937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 18 12:01:46.248937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 18 12:01:46.248937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 18 12:01:46.248937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 18 12:01:46.248937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 18 12:01:46.248937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 18 12:01:46.248937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 18 12:01:46.248937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:46.248937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 18 12:01:46.248937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:46.248937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:46.248937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 18 12:01:46.248937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 18 12:01:46.248937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 18 12:01:46.248937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return ref.to_dict() Aug 18 12:01:46.248937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 18 12:01:46.248937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application next(self.gen) Aug 18 12:01:46.250429 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 18 12:01:46.250429 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application yield resource Aug 18 12:01:46.250429 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 18 12:01:46.250429 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application next(self.gen) Aug 18 12:01:46.250429 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 18 12:01:46.250429 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self.session.rollback() Aug 18 12:01:46.250429 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 18 12:01:46.250429 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self.force_reraise() Aug 18 12:01:46.250429 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 18 12:01:46.250429 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise self.value Aug 18 12:01:46.250429 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 18 12:01:46.250429 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 18 12:01:46.250429 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 18 12:01:46.250429 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application session.commit() Aug 18 12:01:46.250429 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 18 12:01:46.250429 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 18 12:01:46.250429 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 18 12:01:46.251287 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._prepare_impl() Aug 18 12:01:46.251287 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 18 12:01:46.251287 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self.session.flush() Aug 18 12:01:46.251287 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 18 12:01:46.251287 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._flush(objects) Aug 18 12:01:46.251287 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 18 12:01:46.251287 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 18 12:01:46.251287 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 18 12:01:46.251287 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application compat.raise_( Aug 18 12:01:46.251287 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 18 12:01:46.251287 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception Aug 18 12:01:46.251287 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 18 12:01:46.251287 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application flush_context.execute() Aug 18 12:01:46.251287 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 18 12:01:46.251287 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rec.execute(self) Aug 18 12:01:46.251287 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 18 12:01:46.251287 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 18 12:01:46.253880 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 18 12:01:46.253880 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 18 12:01:46.253880 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 18 12:01:46.253880 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 18 12:01:46.253880 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 18 12:01:46.253880 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 18 12:01:46.253880 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 18 12:01:46.253880 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 18 12:01:46.253880 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 18 12:01:46.253880 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 18 12:01:46.253880 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 18 12:01:46.253880 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 18 12:01:46.253880 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 18 12:01:46.253880 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 18 12:01:46.253880 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 18 12:01:46.253880 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception Aug 18 12:01:46.255846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 18 12:01:46.255846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 18 12:01:46.255846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 18 12:01:46.255846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 18 12:01:46.255846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 18 12:01:46.255846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application result = self._query(query) Aug 18 12:01:46.255846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 18 12:01:46.255846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application conn.query(q) Aug 18 12:01:46.255846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 18 12:01:46.255846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 18 12:01:46.255846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 18 12:01:46.255846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application result.read() Aug 18 12:01:46.255846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 18 12:01:46.255846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 18 12:01:46.255846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 18 12:01:46.255846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 18 12:01:46.260833 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 18 12:01:46.260833 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 18 12:01:46.260833 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 18 12:01:46.260833 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 18 12:01:46.260833 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 18 12:01:46.260833 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 18 12:01:46.260833 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [parameters: {'id': '1b710c59b9884804af7af60d2001329f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 18 12:01:46.260833 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 18 12:01:46.260833 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:46.260833 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 12:01:46.260833 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:46.260833 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:46.260833 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:46.260833 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:46.260833 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:46.260833 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:46.262807 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:46.262807 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:46.262807 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:46.262807 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:46.262807 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:46.262807 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:46.262807 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 18 12:01:46.262807 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 18 12:01:46.262807 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:46.262807 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:46.262807 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 18 12:01:46.262807 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 18 12:01:46.262807 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 18 12:01:46.262807 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 18 12:01:46.262807 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 18 12:01:46.262807 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:46.268768 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 239/478] 10.222.0.98 () {60 vars in 1131 bytes} [Thu Aug 18 12:01:45 2022] POST /identity/v3/roles => generated 144 bytes in 311 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:46.268983 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e0139f-67e9-4634-98b0-4d5d16e26e85 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:46.310945 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:46.418747 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e0139f-67e9-4634-98b0-4d5d16e26e85 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:46.473097 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4547424-2276-40b0-94bf-b35841734fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:46.473748 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4547424-2276-40b0-94bf-b35841734fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:46.474033 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4547424-2276-40b0-94bf-b35841734fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:46.474312 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4547424-2276-40b0-94bf-b35841734fb2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:46.496507 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4547424-2276-40b0-94bf-b35841734fb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:46.501379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-a4547424-2276-40b0-94bf-b35841734fb2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 12:01:46.501379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:46.501379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:46.501379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:46.501379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:46.501379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:46.501379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:46.501379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:46.501379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:46.501379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:46.501379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:46.501379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:46.501379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:46.501379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:46.501379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:46.501379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:46.502252 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:46.502252 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:46.502252 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:46.502252 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:46.502252 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:46.502252 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:46.502252 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:46.502252 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:46.502252 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:46.502252 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:46.502252 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:46.502252 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:46.502252 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:46.502252 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:46.502252 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:46.502252 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:46.503057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:46.503057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:46.503057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:46.503057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:46.503057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:46.503057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:46.503057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:46.503057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:46.503057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:46.503057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:46.503057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 18 12:01:46.503057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:46.503057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 240/479] 10.222.0.98 () {56 vars in 1114 bytes} [Thu Aug 18 12:01:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:46.508564 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbba7b2-f999-4607-91b8-bad5edd95687 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:46.652400 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbba7b2-f999-4607-91b8-bad5edd95687 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:46.712639 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d2492b-9466-44db-bc39-459dc3266ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:46.713274 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d2492b-9466-44db-bc39-459dc3266ffe None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:46.713564 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d2492b-9466-44db-bc39-459dc3266ffe None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:46.713839 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d2492b-9466-44db-bc39-459dc3266ffe None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:46.714517 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d2492b-9466-44db-bc39-459dc3266ffe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:46.714832 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d2492b-9466-44db-bc39-459dc3266ffe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:46.716237 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d2492b-9466-44db-bc39-459dc3266ffe None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:46.731147 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 240/480] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:01:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:46.876334 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c74fb2-929a-41ad-affb-c4bc1e29804e None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:46.876845 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c74fb2-929a-41ad-affb-c4bc1e29804e None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:46.876845 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c74fb2-929a-41ad-affb-c4bc1e29804e None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:46.878160 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 241/481] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:46.884092 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab113f1-5785-4506-8105-4d0aa5aeeb7c None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:46.884402 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab113f1-5785-4506-8105-4d0aa5aeeb7c None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:46.884665 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab113f1-5785-4506-8105-4d0aa5aeeb7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:46.913130 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-cab113f1-5785-4506-8105-4d0aa5aeeb7c None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:46.914356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-cab113f1-5785-4506-8105-4d0aa5aeeb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:46.950734 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:47.014852 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 241/482] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 12:01:47.307984 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c74fb2-929a-41ad-affb-c4bc1e29804e None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:47.438170 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:47.453220 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c74fb2-929a-41ad-affb-c4bc1e29804e None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:47.502150 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654fbadd-9a9e-4685-a5eb-22dd7e53eb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:47.502806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654fbadd-9a9e-4685-a5eb-22dd7e53eb83 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:47.503092 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654fbadd-9a9e-4685-a5eb-22dd7e53eb83 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:47.503370 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654fbadd-9a9e-4685-a5eb-22dd7e53eb83 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:47.520735 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654fbadd-9a9e-4685-a5eb-22dd7e53eb83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:47.525873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-654fbadd-9a9e-4685-a5eb-22dd7e53eb83 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:01:47.525873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:47.525873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:47.525873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:47.525873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:47.525873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:47.525873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:47.525873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:47.525873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:47.525873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:47.525873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:47.525873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:47.525873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:47.525873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:47.525873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:47.525873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:47.526795 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:47.526795 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:47.526795 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:47.526795 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:47.526795 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:47.526795 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:47.526795 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:47.526795 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:47.526795 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:47.526795 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:47.526795 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:47.526795 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:47.526795 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:47.526795 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:47.526795 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:47.526795 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:47.527601 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:47.527601 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:47.527601 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:47.527601 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:47.527601 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:47.527601 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:47.527601 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:47.527601 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:47.527601 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:47.527601 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:47.527601 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:01:47.527601 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:47.527601 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 242/483] 10.222.0.98 () {56 vars in 1102 bytes} [Thu Aug 18 12:01:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:47.535681 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab113f1-5785-4506-8105-4d0aa5aeeb7c None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:47.642457 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab113f1-5785-4506-8105-4d0aa5aeeb7c None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:47.679521 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd0d12-8003-4ae9-aded-91b4eb625fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:47.680907 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd0d12-8003-4ae9-aded-91b4eb625fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:47.681389 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd0d12-8003-4ae9-aded-91b4eb625fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:47.681887 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd0d12-8003-4ae9-aded-91b4eb625fb2 None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:47.682761 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd0d12-8003-4ae9-aded-91b4eb625fb2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:47.683263 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd0d12-8003-4ae9-aded-91b4eb625fb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:47.684863 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd0d12-8003-4ae9-aded-91b4eb625fb2 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:47.694986 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 242/484] 10.222.0.98 () {56 vars in 1103 bytes} [Thu Aug 18 12:01:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:47.700257 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654fbadd-9a9e-4685-a5eb-22dd7e53eb83 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:47.810462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea434c7-f29c-426e-b73e-39009617e2fe None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:47.811101 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea434c7-f29c-426e-b73e-39009617e2fe None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:47.811622 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea434c7-f29c-426e-b73e-39009617e2fe None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:47.813784 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 243/485] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 12:01:47.820944 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db2088-4d93-4634-a337-969532b9e1da None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:47.821238 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db2088-4d93-4634-a337-969532b9e1da None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:47.821525 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db2088-4d93-4634-a337-969532b9e1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:47.848377 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654fbadd-9a9e-4685-a5eb-22dd7e53eb83 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:47.858617 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-76db2088-4d93-4634-a337-969532b9e1da None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:47.859890 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-76db2088-4d93-4634-a337-969532b9e1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:47.891156 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:47.899210 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac101b3-453d-4c2a-9830-f2a050465dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:47.900661 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac101b3-453d-4c2a-9830-f2a050465dac None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:47.908322 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac101b3-453d-4c2a-9830-f2a050465dac None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:47.908322 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac101b3-453d-4c2a-9830-f2a050465dac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:47.914820 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac101b3-453d-4c2a-9830-f2a050465dac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:47.916832 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac101b3-453d-4c2a-9830-f2a050465dac None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:47.935790 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-8ac101b3-453d-4c2a-9830-f2a050465dac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:47.935790 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:47.935790 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:47.935790 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:47.935790 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:47.935790 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:47.935790 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:47.935790 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:47.935790 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:47.935790 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:47.935790 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:47.935790 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:47.935790 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:01:47.935790 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:01:47.935790 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:01:47.935790 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:01:47.942419 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:47.942419 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:47.942419 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:47.942419 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:47.942419 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:47.942419 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:47.942419 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:47.942419 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:47.942419 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:47.942419 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:47.942419 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:47.942419 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:47.942419 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:47.942419 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:47.942419 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:47.942419 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:47.942419 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:01:47.943288 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:01:47.943288 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:47.943288 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:47.943288 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 243/486] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:01:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:47.946235 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac101b3-453d-4c2a-9830-f2a050465dac None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:47.949543 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:47.994319 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 244/487] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:48.005614 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d4b75-d249-4bc3-86ff-353e52e2c2d4 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:48.006035 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d4b75-d249-4bc3-86ff-353e52e2c2d4 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:48.006419 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d4b75-d249-4bc3-86ff-353e52e2c2d4 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:48.008133 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 245/488] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 12:01:48.013101 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e539d-7a54-4797-a8a5-a2dfcff40f8a None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:48.013499 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e539d-7a54-4797-a8a5-a2dfcff40f8a None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:48.013882 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e539d-7a54-4797-a8a5-a2dfcff40f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:48.052050 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-1f8e539d-7a54-4797-a8a5-a2dfcff40f8a None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:48.052779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-1f8e539d-7a54-4797-a8a5-a2dfcff40f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:48.126291 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac101b3-453d-4c2a-9830-f2a050465dac None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:48.165357 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c2e58-5415-4025-92dd-52adae80fdf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:48.166296 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c2e58-5415-4025-92dd-52adae80fdf2 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:48.166782 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c2e58-5415-4025-92dd-52adae80fdf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:48.167203 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c2e58-5415-4025-92dd-52adae80fdf2 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:48.168021 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07c2e58-5415-4025-92dd-52adae80fdf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:48.168486 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07c2e58-5415-4025-92dd-52adae80fdf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:48.169124 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 246/489] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:48.170263 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07c2e58-5415-4025-92dd-52adae80fdf2 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:48.171997 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1698fb5-ed37-4c7f-b797-648a2eb2c821 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:48.172586 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1698fb5-ed37-4c7f-b797-648a2eb2c821 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:48.173101 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1698fb5-ed37-4c7f-b797-648a2eb2c821 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:48.175288 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 247/490] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:48.183449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd85594-09b4-4fb8-904a-0e8d4c2d0149 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:48.184018 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd85594-09b4-4fb8-904a-0e8d4c2d0149 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:48.184432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd85594-09b4-4fb8-904a-0e8d4c2d0149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:48.186102 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 244/491] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:01:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:01:48.192641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c2e58-5415-4025-92dd-52adae80fdf2 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:48.223335 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-2dd85594-09b4-4fb8-904a-0e8d4c2d0149 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:48.225862 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-2dd85594-09b4-4fb8-904a-0e8d4c2d0149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:48.285497 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:48.328651 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c2e58-5415-4025-92dd-52adae80fdf2 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:48.374064 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcea03f-89dd-413c-973c-0592026715ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:48.374064 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcea03f-89dd-413c-973c-0592026715ed None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:48.374064 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcea03f-89dd-413c-973c-0592026715ed None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:48.374064 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcea03f-89dd-413c-973c-0592026715ed None admin] PATH_INFO: `/v3/users/nova` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:48.381244 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcea03f-89dd-413c-973c-0592026715ed None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:48.382653 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcea03f-89dd-413c-973c-0592026715ed None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:48.387602 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 248/492] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:48.388522 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd85594-09b4-4fb8-904a-0e8d4c2d0149 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:48.412530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-8fcea03f-89dd-413c-973c-0592026715ed None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 12:01:48.412530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:48.412530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:48.412530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:48.412530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:48.412530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:48.412530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:48.412530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:48.412530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:48.412530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:48.412530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:48.412530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:48.412530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 12:01:48.412530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 12:01:48.412530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 12:01:48.412530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 12:01:48.413476 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:48.413476 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:48.413476 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:01:48.413476 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:01:48.413476 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:01:48.413476 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:01:48.413476 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:48.413476 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:48.413476 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:48.413476 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:48.413476 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:48.413476 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:48.413476 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:48.413476 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:48.413476 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:48.413476 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:48.413476 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:48.413476 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:48.414358 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:48.414358 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:48.414358 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 12:01:48.414358 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 12:01:48.414358 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 12:01:48.414358 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 12:01:48.414358 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 12:01:48.414358 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 12:01:48.414358 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 12:01:48.414358 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:48.415805 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:48.416099 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 245/493] 10.222.0.98 () {56 vars in 1093 bytes} [Thu Aug 18 12:01:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:48.422366 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcea03f-89dd-413c-973c-0592026715ed None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:48.539367 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcea03f-89dd-413c-973c-0592026715ed None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:48.557133 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd85594-09b4-4fb8-904a-0e8d4c2d0149 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:48.563585 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:48.575520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef84ed-d055-447b-8f08-42c871c21ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:48.576371 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef84ed-d055-447b-8f08-42c871c21ceb None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:48.577491 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef84ed-d055-447b-8f08-42c871c21ceb None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:48.577491 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef84ed-d055-447b-8f08-42c871c21ceb None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:48.578070 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edef84ed-d055-447b-8f08-42c871c21ceb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:48.578518 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edef84ed-d055-447b-8f08-42c871c21ceb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:48.580200 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edef84ed-d055-447b-8f08-42c871c21ceb None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:48.598264 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73bfd41-4077-4b62-b74c-9f7ee54c383d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:48.599123 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73bfd41-4077-4b62-b74c-9f7ee54c383d None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:48.599623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73bfd41-4077-4b62-b74c-9f7ee54c383d None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:48.599933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73bfd41-4077-4b62-b74c-9f7ee54c383d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:48.612648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73bfd41-4077-4b62-b74c-9f7ee54c383d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:48.617522 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-b73bfd41-4077-4b62-b74c-9f7ee54c383d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:01:48.617522 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:48.617522 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:48.617522 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:48.617522 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:48.617522 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:48.617522 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:48.617522 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:48.617522 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:48.617522 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:48.617522 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:48.617522 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:48.617522 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:48.617522 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:48.617522 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:48.617522 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:48.618593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:48.618593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:48.618593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:48.618593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:48.618593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:48.618593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:48.618593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:48.618593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:48.618593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:48.618593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:48.618593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:48.618593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:48.618593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:48.618593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:48.618593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:48.618593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:48.619616 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:48.619616 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:48.619616 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:48.619616 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:48.619616 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:48.619616 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:48.619616 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:48.619616 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:48.619616 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:48.619616 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:48.619616 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:01:48.619616 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:48.624985 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 249/494] 10.222.0.98 () {56 vars in 1102 bytes} [Thu Aug 18 12:01:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:48.624985 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73bfd41-4077-4b62-b74c-9f7ee54c383d None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:48.632691 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 246/495] 10.222.0.98 () {56 vars in 1133 bytes} [Thu Aug 18 12:01:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:48.633561 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef84ed-d055-447b-8f08-42c871c21ceb None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:48.715534 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:48.749110 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73bfd41-4077-4b62-b74c-9f7ee54c383d None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:48.766100 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef84ed-d055-447b-8f08-42c871c21ceb None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:48.788241 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db555401-e21a-4cab-9dc7-5a08f45ced4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:48.788873 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db555401-e21a-4cab-9dc7-5a08f45ced4e None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:48.789152 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db555401-e21a-4cab-9dc7-5a08f45ced4e None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:48.789426 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db555401-e21a-4cab-9dc7-5a08f45ced4e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:48.800014 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db555401-e21a-4cab-9dc7-5a08f45ced4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:48.804613 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-db555401-e21a-4cab-9dc7-5a08f45ced4e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:01:48.804613 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:48.804613 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:48.804613 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:48.804613 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:48.804613 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:48.804613 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:48.804613 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:48.804613 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:48.804613 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:48.804613 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:48.804613 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:48.804613 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:48.804613 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:48.804613 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:48.804613 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:48.805655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:48.805655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:48.805655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:48.805655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:48.805655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:48.805655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:48.805655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:48.805655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:48.805655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:48.805655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:48.805655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:48.805655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:48.805655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:48.805655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:48.805655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:48.805655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:48.806575 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:48.806575 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:48.806575 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:48.806575 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:48.806575 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:48.806575 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:48.806575 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:48.806575 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:48.806575 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:48.806575 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:48.806575 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:01:48.806575 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:48.806575 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 250/496] 10.222.0.98 () {56 vars in 1102 bytes} [Thu Aug 18 12:01:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:48.807320 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db555401-e21a-4cab-9dc7-5a08f45ced4e None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:48.810158 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bb012-891b-48d0-9e1a-be8ddf061c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:48.811212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bb012-891b-48d0-9e1a-be8ddf061c24 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:48.811666 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bb012-891b-48d0-9e1a-be8ddf061c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:48.812986 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bb012-891b-48d0-9e1a-be8ddf061c24 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:48.813861 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310bb012-891b-48d0-9e1a-be8ddf061c24 None admin] RBAC: Adding query filter params (name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:48.814309 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310bb012-891b-48d0-9e1a-be8ddf061c24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:48.815879 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310bb012-891b-48d0-9e1a-be8ddf061c24 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:48.831555 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 247/497] 10.222.0.98 () {56 vars in 1103 bytes} [Thu Aug 18 12:01:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:48.831555 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bb012-891b-48d0-9e1a-be8ddf061c24 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:48.878080 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:48.927445 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db555401-e21a-4cab-9dc7-5a08f45ced4e None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:48.942486 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bb012-891b-48d0-9e1a-be8ddf061c24 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:48.964708 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:48.971650 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263492bd-846c-483d-bca7-b3f209faa710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:48.972335 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263492bd-846c-483d-bca7-b3f209faa710 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:48.972534 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263492bd-846c-483d-bca7-b3f209faa710 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:48.972815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263492bd-846c-483d-bca7-b3f209faa710 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:48.980977 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a2201-a067-4393-ba7a-337d94b804a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:48.981726 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a2201-a067-4393-ba7a-337d94b804a4 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:48.981899 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a2201-a067-4393-ba7a-337d94b804a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:48.989271 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a2201-a067-4393-ba7a-337d94b804a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:48.994034 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663a2201-a067-4393-ba7a-337d94b804a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:48.999607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-663a2201-a067-4393-ba7a-337d94b804a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:01:48.999607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:48.999607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:48.999607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:48.999607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:48.999607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:48.999607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:48.999607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:48.999607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:48.999607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:48.999607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:48.999607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:48.999607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:48.999607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:48.999607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:48.999607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:49.000679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:49.000679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:49.000679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:49.000679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:49.000679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:49.000679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:49.000679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:49.000679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:49.000679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:49.000679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:49.000679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:49.000679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:49.000679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:49.000679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:49.000679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:49.000679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:49.001504 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:49.001504 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:49.001504 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:49.001504 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:49.001504 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:49.001504 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:49.001504 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:49.001504 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:49.001504 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:49.001504 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:49.001504 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:01:49.001504 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:49.001504 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 248/498] 10.222.0.98 () {56 vars in 1102 bytes} [Thu Aug 18 12:01:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:49.001504 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a2201-a067-4393-ba7a-337d94b804a4 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:49.050056 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263492bd-846c-483d-bca7-b3f209faa710 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:49.051878 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263492bd-846c-483d-bca7-b3f209faa710 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:49.112239 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a2201-a067-4393-ba7a-337d94b804a4 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:49.113885 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:49.151985 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c379e-de7b-47b4-97d3-964929fc6c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:49.152596 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c379e-de7b-47b4-97d3-964929fc6c92 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:49.152878 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c379e-de7b-47b4-97d3-964929fc6c92 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:49.153152 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c379e-de7b-47b4-97d3-964929fc6c92 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:49.153811 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13c379e-de7b-47b4-97d3-964929fc6c92 None admin] RBAC: Adding query filter params (name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:49.154114 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13c379e-de7b-47b4-97d3-964929fc6c92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:49.155499 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13c379e-de7b-47b4-97d3-964929fc6c92 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:49.162816 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 251/499] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:01:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:01:49.162816 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263492bd-846c-483d-bca7-b3f209faa710 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:49.171167 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 249/500] 10.222.0.98 () {56 vars in 1103 bytes} [Thu Aug 18 12:01:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:49.171167 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c379e-de7b-47b4-97d3-964929fc6c92 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:49.194264 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:49.272301 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263492bd-846c-483d-bca7-b3f209faa710 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:49.276956 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c379e-de7b-47b4-97d3-964929fc6c92 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:49.310428 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf44e6-0745-480b-9177-6eda43762045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:49.311066 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf44e6-0745-480b-9177-6eda43762045 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:49.311351 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf44e6-0745-480b-9177-6eda43762045 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:49.311648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf44e6-0745-480b-9177-6eda43762045 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:49.312524 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a5703-819d-4ce8-bdfb-88da12b320a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:49.313126 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a5703-819d-4ce8-bdfb-88da12b320a5 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:49.313408 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a5703-819d-4ce8-bdfb-88da12b320a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:49.313682 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a5703-819d-4ce8-bdfb-88da12b320a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:49.314379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23a5703-819d-4ce8-bdfb-88da12b320a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:49.314687 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23a5703-819d-4ce8-bdfb-88da12b320a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:49.316092 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23a5703-819d-4ce8-bdfb-88da12b320a5 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:49.323792 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baf44e6-0745-480b-9177-6eda43762045 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:49.329187 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 250/501] 10.222.0.98 () {56 vars in 1103 bytes} [Thu Aug 18 12:01:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:49.329187 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a5703-819d-4ce8-bdfb-88da12b320a5 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:49.329391 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baf44e6-0745-480b-9177-6eda43762045 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:49.344661 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-8baf44e6-0745-480b-9177-6eda43762045 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:49.344661 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:49.344661 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:49.344661 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:49.344661 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:49.344661 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:49.344661 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:49.344661 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:49.344661 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:49.344661 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:49.344661 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:49.344661 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:49.344661 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:01:49.344661 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:01:49.344661 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:01:49.344661 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:01:49.345651 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:49.345651 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:49.345651 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:49.345651 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:49.345651 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:49.345651 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:49.345651 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:49.345651 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:49.345651 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:49.345651 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:49.345651 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:49.345651 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:49.345651 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:49.345651 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:49.345651 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:49.345651 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:49.345651 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:01:49.347283 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:01:49.347283 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:49.347283 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:49.347283 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 252/502] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:01:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:49.347283 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf44e6-0745-480b-9177-6eda43762045 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:49.362322 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:49.434623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:49.442322 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a5703-819d-4ce8-bdfb-88da12b320a5 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:49.454122 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf44e6-0745-480b-9177-6eda43762045 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:49.475682 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77fcc7-67e0-4bd0-b57a-d354c082eeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:49.476314 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77fcc7-67e0-4bd0-b57a-d354c082eeea None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:49.476597 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77fcc7-67e0-4bd0-b57a-d354c082eeea None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:49.476871 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77fcc7-67e0-4bd0-b57a-d354c082eeea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:49.487371 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d77fcc7-67e0-4bd0-b57a-d354c082eeea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:49.488869 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d77fcc7-67e0-4bd0-b57a-d354c082eeea None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:49.488980 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ebe53b-5dcf-4dff-9c3a-cd3f0b6e70bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:49.489496 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebe53b-5dcf-4dff-9c3a-cd3f0b6e70bd None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:49.489780 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebe53b-5dcf-4dff-9c3a-cd3f0b6e70bd None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:49.490053 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebe53b-5dcf-4dff-9c3a-cd3f0b6e70bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:49.501169 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ebe53b-5dcf-4dff-9c3a-cd3f0b6e70bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:49.502628 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ebe53b-5dcf-4dff-9c3a-cd3f0b6e70bd None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:49.504226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-9d77fcc7-67e0-4bd0-b57a-d354c082eeea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:49.504226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:49.504226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:49.504226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:49.504226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:49.504226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:49.504226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:49.504226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:49.504226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:49.504226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:49.504226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:49.504226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:49.504226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:01:49.504226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:01:49.504226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:01:49.504226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:01:49.505330 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:49.505330 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:49.505330 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:49.505330 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:49.505330 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:49.505330 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:49.505330 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:49.505330 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:49.505330 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:49.505330 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:49.505330 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:49.505330 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:49.505330 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:49.505330 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:49.505330 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:49.505330 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:49.505330 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:01:49.506175 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:01:49.506175 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:49.506175 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:49.506175 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 251/503] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:01:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:49.506517 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77fcc7-67e0-4bd0-b57a-d354c082eeea None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:49.521365 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-39ebe53b-5dcf-4dff-9c3a-cd3f0b6e70bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:49.521365 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:49.521365 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:49.521365 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:49.521365 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:49.521365 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:49.521365 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:49.521365 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:49.521365 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:49.521365 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:49.521365 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:49.521365 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:49.521365 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:01:49.521365 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:01:49.521365 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:01:49.521365 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:01:49.525078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:49.525078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:49.525078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:49.525078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:49.525078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:49.525078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:49.525078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:49.525078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:49.525078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:49.525078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:49.525078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:49.525078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:49.525078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:49.525078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:49.525078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:49.525078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:49.525078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:01:49.525945 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:01:49.525945 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:49.525945 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:49.525945 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 253/504] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:01:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:49.525945 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ebe53b-5dcf-4dff-9c3a-cd3f0b6e70bd None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:49.588174 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:49.609596 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77fcc7-67e0-4bd0-b57a-d354c082eeea None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:49.625901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ebe53b-5dcf-4dff-9c3a-cd3f0b6e70bd None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:49.644202 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083f7671-df18-4e6e-94aa-a4302ec92243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:49.644835 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f7671-df18-4e6e-94aa-a4302ec92243 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:49.645130 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f7671-df18-4e6e-94aa-a4302ec92243 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:49.645408 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f7671-df18-4e6e-94aa-a4302ec92243 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:49.656929 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083f7671-df18-4e6e-94aa-a4302ec92243 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:49.658391 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083f7671-df18-4e6e-94aa-a4302ec92243 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:49.662350 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4babbd9f-e528-405c-875d-0154d9fcbe9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:49.662954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babbd9f-e528-405c-875d-0154d9fcbe9e None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:49.663235 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babbd9f-e528-405c-875d-0154d9fcbe9e None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:49.663544 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babbd9f-e528-405c-875d-0154d9fcbe9e None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:49.664180 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4babbd9f-e528-405c-875d-0154d9fcbe9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:49.664481 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4babbd9f-e528-405c-875d-0154d9fcbe9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:49.665848 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4babbd9f-e528-405c-875d-0154d9fcbe9e None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:49.667322 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:49.672730 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-083f7671-df18-4e6e-94aa-a4302ec92243 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:49.672730 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:49.672730 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:49.672730 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:49.672730 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:49.672730 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:49.672730 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:49.672730 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:49.672730 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:49.672730 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:49.672730 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:49.672730 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:49.672730 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:01:49.672730 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:01:49.672730 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:01:49.672730 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:01:49.673642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:49.673642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:49.673642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:49.673642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:49.673642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:49.673642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:49.673642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:49.673642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:49.673642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:49.673642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:49.673642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:49.673642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:49.673642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:49.673642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:49.673642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:49.673642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:49.673642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:01:49.674496 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:01:49.674496 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:49.674496 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:49.674496 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 252/505] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:01:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:49.676665 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200b0e6-0b48-4a64-88c8-7dc3c601a86a None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:49.676951 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200b0e6-0b48-4a64-88c8-7dc3c601a86a None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:49.682885 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200b0e6-0b48-4a64-88c8-7dc3c601a86a None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:49.682885 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 253/506] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:49.682885 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200b0e6-0b48-4a64-88c8-7dc3c601a86a None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:49.683143 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 254/507] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:01:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:49.683143 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4babbd9f-e528-405c-875d-0154d9fcbe9e None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:49.782854 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200b0e6-0b48-4a64-88c8-7dc3c601a86a None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:49.791889 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4babbd9f-e528-405c-875d-0154d9fcbe9e None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:49.811714 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:49.818238 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c218a-4d52-4148-bc95-6e8ce147dacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:49.818851 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c218a-4d52-4148-bc95-6e8ce147dacb None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:49.819136 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c218a-4d52-4148-bc95-6e8ce147dacb None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:49.819439 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c218a-4d52-4148-bc95-6e8ce147dacb None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:49.820082 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7c218a-4d52-4148-bc95-6e8ce147dacb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:49.824592 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7c218a-4d52-4148-bc95-6e8ce147dacb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:49.824592 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7c218a-4d52-4148-bc95-6e8ce147dacb None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:49.827226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d7a98-38e7-41e3-a5ef-4aba3d1dd0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:49.827858 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d7a98-38e7-41e3-a5ef-4aba3d1dd0fe None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:49.828141 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d7a98-38e7-41e3-a5ef-4aba3d1dd0fe None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:49.828417 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d7a98-38e7-41e3-a5ef-4aba3d1dd0fe None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:49.829054 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9d7a98-38e7-41e3-a5ef-4aba3d1dd0fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:49.829357 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9d7a98-38e7-41e3-a5ef-4aba3d1dd0fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:49.830854 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9d7a98-38e7-41e3-a5ef-4aba3d1dd0fe None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:49.833318 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 254/508] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:01:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:49.838072 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c218a-4d52-4148-bc95-6e8ce147dacb None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:49.847281 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 255/509] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:01:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:49.847281 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207f503-9745-4698-a3e0-6339a6ef50fc None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:49.847281 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207f503-9745-4698-a3e0-6339a6ef50fc None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:49.847281 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207f503-9745-4698-a3e0-6339a6ef50fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:49.874078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-0207f503-9745-4698-a3e0-6339a6ef50fc None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:49.875592 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-0207f503-9745-4698-a3e0-6339a6ef50fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:49.927498 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:49.944006 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 256/510] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:49.944278 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c218a-4d52-4148-bc95-6e8ce147dacb None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:49.944694 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207f503-9745-4698-a3e0-6339a6ef50fc None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:49.980090 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74bdad7-b5c7-472a-9a2e-699300bb8716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:49.980685 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74bdad7-b5c7-472a-9a2e-699300bb8716 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:49.980968 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74bdad7-b5c7-472a-9a2e-699300bb8716 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:49.981242 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74bdad7-b5c7-472a-9a2e-699300bb8716 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:49.981885 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74bdad7-b5c7-472a-9a2e-699300bb8716 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:49.982195 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74bdad7-b5c7-472a-9a2e-699300bb8716 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:49.983616 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74bdad7-b5c7-472a-9a2e-699300bb8716 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:50.000466 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 255/511] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:01:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:50.000466 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74bdad7-b5c7-472a-9a2e-699300bb8716 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:50.054610 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:50.063861 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207f503-9745-4698-a3e0-6339a6ef50fc None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:50.103380 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40deb38-e387-4678-8796-8774b2ee532f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:50.104042 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40deb38-e387-4678-8796-8774b2ee532f None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:50.104350 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40deb38-e387-4678-8796-8774b2ee532f None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:50.104598 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40deb38-e387-4678-8796-8774b2ee532f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:50.110135 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74bdad7-b5c7-472a-9a2e-699300bb8716 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:50.113024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40deb38-e387-4678-8796-8774b2ee532f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:50.113940 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40deb38-e387-4678-8796-8774b2ee532f None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:50.127480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-e40deb38-e387-4678-8796-8774b2ee532f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 12:01:50.127480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:50.127480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:50.127480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:50.127480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:50.127480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:50.127480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:50.127480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:50.127480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:50.127480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:50.127480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:50.127480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:50.127480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 12:01:50.127480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 12:01:50.127480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 12:01:50.127480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 12:01:50.128438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:50.128438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:50.128438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:01:50.128438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:01:50.128438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:01:50.128438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:01:50.128438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:50.128438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:50.128438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:50.128438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:50.128438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:50.128438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:50.128438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:50.128438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:50.128438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:50.128438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:50.128438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:50.128438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:50.129333 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:50.129333 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:50.129333 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 12:01:50.129333 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 12:01:50.129333 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 12:01:50.129333 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 12:01:50.129333 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 12:01:50.129333 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 12:01:50.129333 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 12:01:50.129333 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:50.129333 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 257/512] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:01:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:50.129905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40deb38-e387-4678-8796-8774b2ee532f None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:50.150784 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b327b153-385e-4d00-b7c6-72cf98b6347a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:50.151404 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327b153-385e-4d00-b7c6-72cf98b6347a None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:50.151722 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327b153-385e-4d00-b7c6-72cf98b6347a None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:50.151997 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327b153-385e-4d00-b7c6-72cf98b6347a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:50.158797 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b327b153-385e-4d00-b7c6-72cf98b6347a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:50.160254 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b327b153-385e-4d00-b7c6-72cf98b6347a None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:50.172315 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-b327b153-385e-4d00-b7c6-72cf98b6347a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:01:50.172315 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:50.172315 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:50.172315 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:50.172315 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:50.172315 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:50.172315 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:50.172315 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:50.172315 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:50.172315 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:50.172315 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:50.172315 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:50.172315 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 12:01:50.172315 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 12:01:50.172315 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 12:01:50.173360 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 12:01:50.173360 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:50.173360 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:50.173360 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:50.173360 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:50.173360 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:50.173360 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:50.173360 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:50.173360 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:50.173360 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:50.173360 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:50.173360 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:50.173360 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:50.173360 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:50.173360 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:50.173360 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:50.173360 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:50.174480 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 12:01:50.174480 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 12:01:50.174480 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:01:50.174480 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:01:50.174480 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:01:50.174480 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:01:50.174480 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:01:50.174480 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:50.177904 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 256/513] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:01:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:50.178753 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b327b153-385e-4d00-b7c6-72cf98b6347a None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:50.183399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:50.241433 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40deb38-e387-4678-8796-8774b2ee532f None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:50.275644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda9669-3eaa-41c0-bb55-dc74b9cde9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:50.276276 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda9669-3eaa-41c0-bb55-dc74b9cde9f6 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:50.276562 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda9669-3eaa-41c0-bb55-dc74b9cde9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:50.276844 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda9669-3eaa-41c0-bb55-dc74b9cde9f6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:50.278362 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b327b153-385e-4d00-b7c6-72cf98b6347a None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:50.284334 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcda9669-3eaa-41c0-bb55-dc74b9cde9f6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:50.285794 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcda9669-3eaa-41c0-bb55-dc74b9cde9f6 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:50.288746 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:50.297647 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-dcda9669-3eaa-41c0-bb55-dc74b9cde9f6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 18 12:01:50.297647 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:50.297647 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:50.297647 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:50.297647 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:50.297647 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:50.297647 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:50.297647 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:50.297647 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:50.297647 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:50.297647 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:50.297647 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:50.297647 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 12:01:50.297647 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 12:01:50.297647 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 12:01:50.297647 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 12:01:50.298556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:50.298556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:50.298556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:01:50.298556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:01:50.298556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:01:50.298556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:01:50.298556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:50.298556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:50.298556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:50.298556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:50.298556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:50.298556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:50.298556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:50.298556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:50.298556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:50.298556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:50.298556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:50.298556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:50.299811 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:50.299811 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:50.299811 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 12:01:50.299811 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 12:01:50.299811 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 12:01:50.299811 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 12:01:50.299811 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 12:01:50.299811 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 12:01:50.299811 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 18 12:01:50.299811 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:50.299811 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 258/514] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:01:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:50.299811 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda9669-3eaa-41c0-bb55-dc74b9cde9f6 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:50.316450 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d9d76f-dc86-4ef6-9112-8368160580ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:50.316450 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9d76f-dc86-4ef6-9112-8368160580ac None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:50.316450 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9d76f-dc86-4ef6-9112-8368160580ac None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:50.316450 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9d76f-dc86-4ef6-9112-8368160580ac None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:50.321731 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d9d76f-dc86-4ef6-9112-8368160580ac None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:50.323173 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d9d76f-dc86-4ef6-9112-8368160580ac None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:50.343270 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-42d9d76f-dc86-4ef6-9112-8368160580ac None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 18 12:01:50.343270 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:50.343270 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:50.343270 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:50.343270 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:50.343270 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:50.343270 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:50.343270 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:50.343270 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:50.343270 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:50.343270 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:50.343270 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:50.343270 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 12:01:50.343270 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 12:01:50.343270 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 12:01:50.343270 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 12:01:50.344437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:50.344437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:50.344437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:01:50.344437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:01:50.344437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:01:50.344437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:01:50.344437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:50.344437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:50.344437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:50.344437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:50.344437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:50.344437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:50.344437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:50.344437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:50.344437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:50.344437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:50.344437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:50.344437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:50.345321 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:50.345321 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:50.345321 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 12:01:50.345321 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 12:01:50.345321 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 12:01:50.345321 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 12:01:50.345321 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 12:01:50.345321 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 12:01:50.345321 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 18 12:01:50.345321 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:50.345321 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 257/515] 10.222.0.98 () {56 vars in 1102 bytes} [Thu Aug 18 12:01:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:50.346708 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d9d76f-dc86-4ef6-9112-8368160580ac None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:50.405682 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda9669-3eaa-41c0-bb55-dc74b9cde9f6 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:50.408549 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:50.439079 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434f3a1-5cc9-4521-bb4d-83d909a86d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:50.439707 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434f3a1-5cc9-4521-bb4d-83d909a86d49 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:50.439991 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434f3a1-5cc9-4521-bb4d-83d909a86d49 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:50.440266 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434f3a1-5cc9-4521-bb4d-83d909a86d49 None admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:50.440987 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6434f3a1-5cc9-4521-bb4d-83d909a86d49 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:50.441300 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6434f3a1-5cc9-4521-bb4d-83d909a86d49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:50.442727 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6434f3a1-5cc9-4521-bb4d-83d909a86d49 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:50.450417 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d9d76f-dc86-4ef6-9112-8368160580ac None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:50.479930 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 259/516] 10.222.0.98 () {56 vars in 1137 bytes} [Thu Aug 18 12:01:50 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:50.480790 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434f3a1-5cc9-4521-bb4d-83d909a86d49 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:50.486682 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5badaaa5-1812-46e5-b86e-05ab86e92bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:50.487373 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5badaaa5-1812-46e5-b86e-05ab86e92bae None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:50.487692 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5badaaa5-1812-46e5-b86e-05ab86e92bae None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:50.487970 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5badaaa5-1812-46e5-b86e-05ab86e92bae None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:50.488672 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5badaaa5-1812-46e5-b86e-05ab86e92bae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:50.488985 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5badaaa5-1812-46e5-b86e-05ab86e92bae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:50.490417 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5badaaa5-1812-46e5-b86e-05ab86e92bae None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:50.508546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 258/517] 10.222.0.98 () {56 vars in 1148 bytes} [Thu Aug 18 12:01:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:50.509476 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5badaaa5-1812-46e5-b86e-05ab86e92bae None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:50.550733 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:50.602757 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434f3a1-5cc9-4521-bb4d-83d909a86d49 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:50.621776 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5badaaa5-1812-46e5-b86e-05ab86e92bae None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:50.636649 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c98ad-c67b-44a8-ad85-5fa3f1352160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:50.637281 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c98ad-c67b-44a8-ad85-5fa3f1352160 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:50.637561 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c98ad-c67b-44a8-ad85-5fa3f1352160 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:50.637837 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c98ad-c67b-44a8-ad85-5fa3f1352160 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:50.648826 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377c98ad-c67b-44a8-ad85-5fa3f1352160 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:50.650278 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377c98ad-c67b-44a8-ad85-5fa3f1352160 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:50.656505 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e47073-bfb0-4d79-8064-12a449774f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:50.657106 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e47073-bfb0-4d79-8064-12a449774f5a None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:50.657388 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e47073-bfb0-4d79-8064-12a449774f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:50.657661 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e47073-bfb0-4d79-8064-12a449774f5a None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:50.658385 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e47073-bfb0-4d79-8064-12a449774f5a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:50.658692 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e47073-bfb0-4d79-8064-12a449774f5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:50.660135 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e47073-bfb0-4d79-8064-12a449774f5a None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:50.664453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-377c98ad-c67b-44a8-ad85-5fa3f1352160 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:50.664453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:50.664453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:50.664453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:50.664453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:50.664453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:50.664453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:50.664453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:50.664453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:50.664453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:50.664453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:50.664453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:50.664453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:01:50.664453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:01:50.664453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:01:50.664453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:01:50.665509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:50.665509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:50.665509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:50.665509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:50.665509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:50.665509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:50.665509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:50.665509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:50.665509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:50.665509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:50.665509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:50.665509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:50.665509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:50.665509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:50.665509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:50.665509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:50.665509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:01:50.668379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:01:50.668379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:50.668379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:50.668379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 260/518] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:01:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:50.668379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c98ad-c67b-44a8-ad85-5fa3f1352160 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:50.678767 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 259/519] 10.222.0.98 () {56 vars in 1137 bytes} [Thu Aug 18 12:01:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:50.679633 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e47073-bfb0-4d79-8064-12a449774f5a None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:50.686433 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:50.771190 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:50.772597 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c98ad-c67b-44a8-ad85-5fa3f1352160 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:50.782961 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e47073-bfb0-4d79-8064-12a449774f5a None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:50.807520 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59591dab-3e6e-4182-8c7a-a18e183c83c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:50.808250 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59591dab-3e6e-4182-8c7a-a18e183c83c0 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:50.808538 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59591dab-3e6e-4182-8c7a-a18e183c83c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:50.808818 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59591dab-3e6e-4182-8c7a-a18e183c83c0 None admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:50.809557 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59591dab-3e6e-4182-8c7a-a18e183c83c0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:50.809865 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59591dab-3e6e-4182-8c7a-a18e183c83c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:50.811291 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59591dab-3e6e-4182-8c7a-a18e183c83c0 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:50.818361 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d893e1-86dd-46f0-a161-9b560c358339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:50.818969 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d893e1-86dd-46f0-a161-9b560c358339 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:50.819248 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d893e1-86dd-46f0-a161-9b560c358339 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:50.819569 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d893e1-86dd-46f0-a161-9b560c358339 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:50.829724 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 261/520] 10.222.0.98 () {56 vars in 1139 bytes} [Thu Aug 18 12:01:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:50.830562 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59591dab-3e6e-4182-8c7a-a18e183c83c0 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:50.906957 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:50.916642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d893e1-86dd-46f0-a161-9b560c358339 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:50.918187 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d893e1-86dd-46f0-a161-9b560c358339 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:50.939019 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59591dab-3e6e-4182-8c7a-a18e183c83c0 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:50.973442 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321aea6-968e-429d-89a3-998f3be1641f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:50.974052 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321aea6-968e-429d-89a3-998f3be1641f None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:50.974335 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321aea6-968e-429d-89a3-998f3be1641f None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:50.974611 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321aea6-968e-429d-89a3-998f3be1641f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:50.975339 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b321aea6-968e-429d-89a3-998f3be1641f None admin] RBAC: Adding query filter params (role.id=563e3ed5c6164aae966b64cc302b114a, scope.project.id=6f676a5169464d83af4267db25025728, user.id=171c72a6b29e418bbb4ea5c6faffe3b3) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:50.975687 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b321aea6-968e-429d-89a3-998f3be1641f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:50.977117 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b321aea6-968e-429d-89a3-998f3be1641f None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:50.982727 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 262/521] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:01:50 2022] GET /identity/v3/role_assignments?user.id=171c72a6b29e418bbb4ea5c6faffe3b3&scope.project.id=6f676a5169464d83af4267db25025728&role.id=563e3ed5c6164aae966b64cc302b114a => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:50.983476 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321aea6-968e-429d-89a3-998f3be1641f None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:51.015663 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:51.033838 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 260/522] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:01:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:01:51.033838 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d893e1-86dd-46f0-a161-9b560c358339 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:51.118729 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321aea6-968e-429d-89a3-998f3be1641f None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:51.147151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d893e1-86dd-46f0-a161-9b560c358339 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:51.156802 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea0ffe-b096-460b-b8cd-ca647c738380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:51.157485 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea0ffe-b096-460b-b8cd-ca647c738380 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:51.157736 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea0ffe-b096-460b-b8cd-ca647c738380 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:51.157985 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea0ffe-b096-460b-b8cd-ca647c738380 None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:51.158632 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea0ffe-b096-460b-b8cd-ca647c738380 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:51.158951 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea0ffe-b096-460b-b8cd-ca647c738380 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:51.160360 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea0ffe-b096-460b-b8cd-ca647c738380 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:51.163238 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:51.174658 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 263/523] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:51.176638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea0ffe-b096-460b-b8cd-ca647c738380 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:51.183941 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b9235e-ce67-41df-82a4-c892dd494ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:51.184963 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9235e-ce67-41df-82a4-c892dd494ebf None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:51.185506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9235e-ce67-41df-82a4-c892dd494ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:51.185991 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9235e-ce67-41df-82a4-c892dd494ebf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:51.249319 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:51.263113 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b9235e-ce67-41df-82a4-c892dd494ebf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:51.265401 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b9235e-ce67-41df-82a4-c892dd494ebf None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:51.297927 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea0ffe-b096-460b-b8cd-ca647c738380 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:51.333683 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6369fa3-d4bb-4143-9bc1-8c6447cd0078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:51.334300 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6369fa3-d4bb-4143-9bc1-8c6447cd0078 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:51.334589 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6369fa3-d4bb-4143-9bc1-8c6447cd0078 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:51.334865 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6369fa3-d4bb-4143-9bc1-8c6447cd0078 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:51.380024 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 261/524] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:01:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:01:51.380024 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b9235e-ce67-41df-82a4-c892dd494ebf None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:51.402346 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:51.417459 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6369fa3-d4bb-4143-9bc1-8c6447cd0078 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:51.417459 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6369fa3-d4bb-4143-9bc1-8c6447cd0078 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:51.484545 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:51.497079 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b9235e-ce67-41df-82a4-c892dd494ebf None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:51.523374 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 264/525] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:01:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:01:51.524294 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6369fa3-d4bb-4143-9bc1-8c6447cd0078 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:51.537442 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29438acc-c56b-43cf-80a0-3c9f6e0059b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:51.538363 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29438acc-c56b-43cf-80a0-3c9f6e0059b5 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:51.538649 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29438acc-c56b-43cf-80a0-3c9f6e0059b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:51.538924 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29438acc-c56b-43cf-80a0-3c9f6e0059b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:51.551903 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29438acc-c56b-43cf-80a0-3c9f6e0059b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:51.553531 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29438acc-c56b-43cf-80a0-3c9f6e0059b5 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:51.569378 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-29438acc-c56b-43cf-80a0-3c9f6e0059b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:51.569378 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:51.569378 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:51.569378 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:51.569378 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:51.569378 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:51.569378 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:51.569378 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:51.569378 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:51.569378 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:51.569378 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:51.569378 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:51.569378 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:01:51.569378 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:01:51.569378 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:01:51.569378 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:01:51.570517 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:51.570517 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:51.570517 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:51.570517 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:51.570517 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:51.570517 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:51.570517 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:51.570517 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:51.570517 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:51.570517 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:51.570517 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:51.570517 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:51.570517 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:51.570517 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:51.570517 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:51.570517 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:51.570517 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:01:51.571587 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:01:51.571587 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:51.571587 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:51.572077 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 262/526] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:01:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:51.572603 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29438acc-c56b-43cf-80a0-3c9f6e0059b5 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:51.635466 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:51.642795 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6369fa3-d4bb-4143-9bc1-8c6447cd0078 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:51.682357 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29438acc-c56b-43cf-80a0-3c9f6e0059b5 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:51.683336 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651fe82e-01e8-49b9-8e39-3f04f42ec900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:51.684095 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651fe82e-01e8-49b9-8e39-3f04f42ec900 None admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:51.684412 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651fe82e-01e8-49b9-8e39-3f04f42ec900 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:51.684769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651fe82e-01e8-49b9-8e39-3f04f42ec900 None admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:51.685417 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651fe82e-01e8-49b9-8e39-3f04f42ec900 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:51.686843 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651fe82e-01e8-49b9-8e39-3f04f42ec900 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:51.721826 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8d375-cfc8-4893-af3c-3896573dcda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:51.722449 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8d375-cfc8-4893-af3c-3896573dcda1 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:51.722733 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8d375-cfc8-4893-af3c-3896573dcda1 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:51.723011 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8d375-cfc8-4893-af3c-3896573dcda1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:51.731684 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:51.733898 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e8d375-cfc8-4893-af3c-3896573dcda1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:51.735358 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e8d375-cfc8-4893-af3c-3896573dcda1 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:51.749098 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-23e8d375-cfc8-4893-af3c-3896573dcda1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:51.749098 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:51.749098 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:51.749098 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:51.749098 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:51.749098 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:51.749098 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:51.749098 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:51.749098 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:51.749098 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:51.749098 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:51.749098 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:51.749098 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:01:51.749098 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:01:51.749098 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:01:51.749098 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:01:51.750212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:51.750212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:51.750212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:51.750212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:51.750212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:51.750212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:51.750212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:51.750212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:51.750212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:51.750212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:51.750212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:51.750212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:51.750212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:51.750212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:51.750212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:51.750212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:51.750212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:01:51.755870 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:01:51.755870 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:51.755870 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:51.755870 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 263/527] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:01:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:51.755870 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8d375-cfc8-4893-af3c-3896573dcda1 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:51.783447 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 265/528] 10.222.0.98 () {60 vars in 1132 bytes} [Thu Aug 18 12:01:51 2022] POST /identity/v3/users => generated 247 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:01:51.783447 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651fe82e-01e8-49b9-8e39-3f04f42ec900 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:51.871317 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8d375-cfc8-4893-af3c-3896573dcda1 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:51.916652 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a792b28-f801-48ae-bdf2-264f22aeafe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:51.917500 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a792b28-f801-48ae-bdf2-264f22aeafe6 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:51.917977 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a792b28-f801-48ae-bdf2-264f22aeafe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:51.918260 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a792b28-f801-48ae-bdf2-264f22aeafe6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:51.928328 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651fe82e-01e8-49b9-8e39-3f04f42ec900 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:51.931449 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a792b28-f801-48ae-bdf2-264f22aeafe6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:51.935468 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a792b28-f801-48ae-bdf2-264f22aeafe6 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:51.949679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-3a792b28-f801-48ae-bdf2-264f22aeafe6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:51.949679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:51.949679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:51.949679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:51.949679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:51.949679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:51.949679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:51.949679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:51.949679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:51.949679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:51.949679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:51.949679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:51.949679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:01:51.949679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:01:51.949679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:01:51.949679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:01:51.950713 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:51.950713 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:51.950713 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:51.950713 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:51.950713 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:51.950713 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:51.950713 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:51.950713 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:51.950713 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:51.950713 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:51.950713 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:51.950713 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:51.950713 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:51.950713 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:51.950713 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:51.950713 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:51.950713 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:01:51.951588 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:01:51.951588 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:51.951588 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:51.951588 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 264/529] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:01:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:51.952261 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a792b28-f801-48ae-bdf2-264f22aeafe6 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:51.974181 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0158cc0-6324-4503-98ad-534afe30855f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:51.975170 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0158cc0-6324-4503-98ad-534afe30855f None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:51.975548 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0158cc0-6324-4503-98ad-534afe30855f None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:51.975893 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0158cc0-6324-4503-98ad-534afe30855f None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:51.976624 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0158cc0-6324-4503-98ad-534afe30855f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:51.977001 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0158cc0-6324-4503-98ad-534afe30855f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:51.978392 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0158cc0-6324-4503-98ad-534afe30855f None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:51.992399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 266/530] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:51.993253 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0158cc0-6324-4503-98ad-534afe30855f None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:52.008634 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:52.012971 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:52.091442 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a792b28-f801-48ae-bdf2-264f22aeafe6 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:52.122387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0158cc0-6324-4503-98ad-534afe30855f None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:52.129939 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1dcf00-7bdc-45b2-a03f-81547526cbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:52.130554 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1dcf00-7bdc-45b2-a03f-81547526cbb4 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:52.130833 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1dcf00-7bdc-45b2-a03f-81547526cbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:52.131104 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1dcf00-7bdc-45b2-a03f-81547526cbb4 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:52.131783 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1dcf00-7bdc-45b2-a03f-81547526cbb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:52.132094 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1dcf00-7bdc-45b2-a03f-81547526cbb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:52.134057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1dcf00-7bdc-45b2-a03f-81547526cbb4 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:52.150660 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 265/531] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:52.151808 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1dcf00-7bdc-45b2-a03f-81547526cbb4 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:52.165670 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4831717f-d460-436c-9a42-be7bcc96a528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:52.166400 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831717f-d460-436c-9a42-be7bcc96a528 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:52.167154 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831717f-d460-436c-9a42-be7bcc96a528 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:52.167555 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831717f-d460-436c-9a42-be7bcc96a528 None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:52.168595 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4831717f-d460-436c-9a42-be7bcc96a528 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:52.168968 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4831717f-d460-436c-9a42-be7bcc96a528 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:52.171187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4831717f-d460-436c-9a42-be7bcc96a528 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:52.186547 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 267/532] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:52.188172 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4831717f-d460-436c-9a42-be7bcc96a528 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:52.279370 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1dcf00-7bdc-45b2-a03f-81547526cbb4 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:52.279685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:52.283555 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:52.319336 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62411e04-673e-4b93-9a65-dc9623fbe215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:52.319997 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62411e04-673e-4b93-9a65-dc9623fbe215 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:52.320276 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62411e04-673e-4b93-9a65-dc9623fbe215 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:52.320543 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62411e04-673e-4b93-9a65-dc9623fbe215 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:52.321164 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4831717f-d460-436c-9a42-be7bcc96a528 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:52.327570 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62411e04-673e-4b93-9a65-dc9623fbe215 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:52.329628 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62411e04-673e-4b93-9a65-dc9623fbe215 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:52.341015 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-62411e04-673e-4b93-9a65-dc9623fbe215 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:01:52.341015 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:52.341015 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:52.341015 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:52.341015 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:52.341015 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:52.341015 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:52.341015 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:52.341015 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:52.341015 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:52.341015 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:52.341015 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:52.341015 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 12:01:52.341015 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 12:01:52.341015 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 12:01:52.341967 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 12:01:52.341967 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:52.341967 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:52.341967 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:52.341967 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:52.341967 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:52.341967 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:52.341967 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:52.341967 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:52.341967 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:52.341967 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:52.341967 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:52.341967 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:52.341967 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:52.341967 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:52.341967 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:52.341967 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:52.342806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 12:01:52.342806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 12:01:52.342806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:01:52.342806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:01:52.342806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:01:52.342806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:01:52.342806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:01:52.342806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:52.342806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 266/533] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:01:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:52.343784 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62411e04-673e-4b93-9a65-dc9623fbe215 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:52.368522 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1264af08-454c-48f0-96fc-4d5ee025cd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:52.369169 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264af08-454c-48f0-96fc-4d5ee025cd19 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:52.369458 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264af08-454c-48f0-96fc-4d5ee025cd19 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:52.369877 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264af08-454c-48f0-96fc-4d5ee025cd19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:52.378074 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1264af08-454c-48f0-96fc-4d5ee025cd19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:52.379575 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1264af08-454c-48f0-96fc-4d5ee025cd19 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:52.393437 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-1264af08-454c-48f0-96fc-4d5ee025cd19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:01:52.393437 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:52.393437 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:52.393437 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:52.393437 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:52.393437 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:52.393437 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:52.393437 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:52.393437 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:52.393437 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:52.393437 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:52.393437 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:52.393437 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 12:01:52.393437 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 12:01:52.393437 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 12:01:52.394329 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 12:01:52.394329 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:52.394329 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:52.394329 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:52.394329 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:52.394329 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:52.394329 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:52.394329 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:52.394329 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:52.394329 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:52.394329 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:52.394329 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:52.394329 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:52.394329 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:52.394329 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:52.394329 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:52.394329 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:52.395178 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 12:01:52.395178 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 12:01:52.395178 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:01:52.395178 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:01:52.395178 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:01:52.395178 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:01:52.395178 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:01:52.395178 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:52.399483 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 268/534] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:01:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:52.400367 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1264af08-454c-48f0-96fc-4d5ee025cd19 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:52.481302 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62411e04-673e-4b93-9a65-dc9623fbe215 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:52.524159 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c128854e-5034-41cf-9b94-60f26ac55b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:52.524363 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1264af08-454c-48f0-96fc-4d5ee025cd19 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:52.524899 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128854e-5034-41cf-9b94-60f26ac55b0f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:52.525276 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128854e-5034-41cf-9b94-60f26ac55b0f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:52.525724 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128854e-5034-41cf-9b94-60f26ac55b0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:52.533538 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c128854e-5034-41cf-9b94-60f26ac55b0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:52.535286 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c128854e-5034-41cf-9b94-60f26ac55b0f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:52.548278 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-c128854e-5034-41cf-9b94-60f26ac55b0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:01:52.548278 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:52.548278 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:52.548278 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:52.548278 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:52.548278 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:52.548278 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:52.548278 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:52.548278 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:52.548278 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:52.548278 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:52.548278 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:52.548278 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 12:01:52.548278 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 12:01:52.548278 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 12:01:52.549164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 12:01:52.549164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:52.549164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:52.549164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:52.549164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:52.549164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:52.549164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:52.549164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:52.549164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:52.549164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:52.549164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:52.549164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:52.549164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:52.549164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:52.549164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:52.549164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:52.549164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:52.550015 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 12:01:52.550015 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 12:01:52.550015 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:01:52.550015 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:01:52.550015 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:01:52.550015 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:01:52.550015 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:01:52.550015 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:52.550015 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 267/535] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:01:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:52.551478 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c128854e-5034-41cf-9b94-60f26ac55b0f None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:52.555736 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:52.560941 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ba1e6-d210-452b-a96c-6802cc9647a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:52.561619 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ba1e6-d210-452b-a96c-6802cc9647a1 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:52.561905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ba1e6-d210-452b-a96c-6802cc9647a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:52.562242 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ba1e6-d210-452b-a96c-6802cc9647a1 None admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:52.563512 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ba1e6-d210-452b-a96c-6802cc9647a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:52.563837 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ba1e6-d210-452b-a96c-6802cc9647a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:52.565412 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ba1e6-d210-452b-a96c-6802cc9647a1 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:52.582064 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:52.583972 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 269/536] 10.222.0.98 () {56 vars in 1148 bytes} [Thu Aug 18 12:01:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:52.584832 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ba1e6-d210-452b-a96c-6802cc9647a1 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:52.674657 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c128854e-5034-41cf-9b94-60f26ac55b0f None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:52.703747 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ba1e6-d210-452b-a96c-6802cc9647a1 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:52.718189 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba317d72-77b8-4e3a-b20b-1eece6551965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:52.719313 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba317d72-77b8-4e3a-b20b-1eece6551965 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:52.719630 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba317d72-77b8-4e3a-b20b-1eece6551965 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:52.719910 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba317d72-77b8-4e3a-b20b-1eece6551965 None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:52.720859 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba317d72-77b8-4e3a-b20b-1eece6551965 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:52.721192 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba317d72-77b8-4e3a-b20b-1eece6551965 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:52.722878 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba317d72-77b8-4e3a-b20b-1eece6551965 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:52.739694 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 268/537] 10.222.0.98 () {56 vars in 1148 bytes} [Thu Aug 18 12:01:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:52.740460 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba317d72-77b8-4e3a-b20b-1eece6551965 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:52.742569 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d13067-4e9e-4c65-a3ce-296471afef09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:52.743533 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d13067-4e9e-4c65-a3ce-296471afef09 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:52.744035 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d13067-4e9e-4c65-a3ce-296471afef09 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:52.744527 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d13067-4e9e-4c65-a3ce-296471afef09 None admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:52.745424 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d13067-4e9e-4c65-a3ce-296471afef09 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:52.745937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d13067-4e9e-4c65-a3ce-296471afef09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:52.747632 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d13067-4e9e-4c65-a3ce-296471afef09 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:52.761979 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 270/538] 10.222.0.98 () {56 vars in 1148 bytes} [Thu Aug 18 12:01:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:52.763312 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d13067-4e9e-4c65-a3ce-296471afef09 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:52.824054 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:52.861108 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:52.879776 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba317d72-77b8-4e3a-b20b-1eece6551965 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:52.883748 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d13067-4e9e-4c65-a3ce-296471afef09 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:52.918180 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f99c56d-2169-4101-b601-fa3550cdb3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:52.918806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99c56d-2169-4101-b601-fa3550cdb3ad None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:52.919091 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99c56d-2169-4101-b601-fa3550cdb3ad None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:52.919369 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99c56d-2169-4101-b601-fa3550cdb3ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:52.920324 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f99c56d-2169-4101-b601-fa3550cdb3ad None admin] RBAC: Adding query filter params (role.id=563e3ed5c6164aae966b64cc302b114a, scope.project.id=6f676a5169464d83af4267db25025728, user.id=c41b6fb3cb064a49b9236077a8faed54) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:52.920648 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f99c56d-2169-4101-b601-fa3550cdb3ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:52.923464 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f99c56d-2169-4101-b601-fa3550cdb3ad None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:52.931109 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 269/539] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:01:52 2022] GET /identity/v3/role_assignments?user.id=c41b6fb3cb064a49b9236077a8faed54&scope.project.id=6f676a5169464d83af4267db25025728&role.id=563e3ed5c6164aae966b64cc302b114a => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:52.931934 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f99c56d-2169-4101-b601-fa3550cdb3ad None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:52.932371 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca67daaa-8f7b-4ae7-a83c-c9ab9fe2e879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:52.933604 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67daaa-8f7b-4ae7-a83c-c9ab9fe2e879 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:52.934128 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67daaa-8f7b-4ae7-a83c-c9ab9fe2e879 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:52.934975 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67daaa-8f7b-4ae7-a83c-c9ab9fe2e879 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:52.937132 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca67daaa-8f7b-4ae7-a83c-c9ab9fe2e879 None admin] RBAC: Adding query filter params (role.id=563e3ed5c6164aae966b64cc302b114a, scope.project.id=6f676a5169464d83af4267db25025728, user.id=d020063702b0481e9f4111680766f7fc) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:52.937662 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca67daaa-8f7b-4ae7-a83c-c9ab9fe2e879 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:52.939507 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca67daaa-8f7b-4ae7-a83c-c9ab9fe2e879 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:52.947943 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 271/540] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:01:52 2022] GET /identity/v3/role_assignments?user.id=d020063702b0481e9f4111680766f7fc&scope.project.id=6f676a5169464d83af4267db25025728&role.id=563e3ed5c6164aae966b64cc302b114a => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:53.011897 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a0d75-34aa-4b93-b84a-1c7c12d897b6 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:53.012092 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a0d75-34aa-4b93-b84a-1c7c12d897b6 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:53.012493 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a0d75-34aa-4b93-b84a-1c7c12d897b6 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:53.014240 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 272/541] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 12:01:53.020480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71241a-5b70-461e-ae60-6d428bd8d309 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:53.020855 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71241a-5b70-461e-ae60-6d428bd8d309 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:53.021235 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71241a-5b70-461e-ae60-6d428bd8d309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:53.041587 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-2f71241a-5b70-461e-ae60-6d428bd8d309 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:53.043078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-2f71241a-5b70-461e-ae60-6d428bd8d309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:53.062562 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f99c56d-2169-4101-b601-fa3550cdb3ad None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:53.115063 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2a00a-ffca-4e8d-a1d1-abfa4453032c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:53.116687 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2a00a-ffca-4e8d-a1d1-abfa4453032c None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:53.117196 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2a00a-ffca-4e8d-a1d1-abfa4453032c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:53.117653 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2a00a-ffca-4e8d-a1d1-abfa4453032c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:53.118751 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a2a00a-ffca-4e8d-a1d1-abfa4453032c None admin] RBAC: Adding query filter params (role.id=563e3ed5c6164aae966b64cc302b114a, scope.project.id=6f676a5169464d83af4267db25025728, user.id=185644affb284c2b91e604cf99803ffb) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:53.119259 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a2a00a-ffca-4e8d-a1d1-abfa4453032c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:53.121074 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a2a00a-ffca-4e8d-a1d1-abfa4453032c None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:53.128423 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 270/542] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:01:52 2022] GET /identity/v3/role_assignments?user.id=185644affb284c2b91e604cf99803ffb&scope.project.id=6f676a5169464d83af4267db25025728&role.id=563e3ed5c6164aae966b64cc302b114a => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:53.138729 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 273/543] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:53.498845 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2a00a-ffca-4e8d-a1d1-abfa4453032c None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:53.519720 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:53.619844 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2a00a-ffca-4e8d-a1d1-abfa4453032c None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:53.668715 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04751727-0f5d-4a10-878a-bc9447f4bf38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:53.669658 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04751727-0f5d-4a10-878a-bc9447f4bf38 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:53.670120 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04751727-0f5d-4a10-878a-bc9447f4bf38 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:53.670571 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04751727-0f5d-4a10-878a-bc9447f4bf38 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:53.687166 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04751727-0f5d-4a10-878a-bc9447f4bf38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:53.692828 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-04751727-0f5d-4a10-878a-bc9447f4bf38 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:01:53.692828 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:53.692828 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:53.692828 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:53.692828 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:53.692828 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:53.692828 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:53.692828 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:53.692828 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:53.692828 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:53.692828 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:53.692828 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:53.692828 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:53.692828 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:53.692828 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:53.692828 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:53.693737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:53.693737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:53.693737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:53.693737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:53.693737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:53.693737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:53.693737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:53.693737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:53.693737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:53.693737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:53.693737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:53.693737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:53.693737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:53.693737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:53.693737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:53.693737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:53.695057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:53.695057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:53.695057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:53.695057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:53.695057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:53.695057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:53.695057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:53.695057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:53.695057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:53.695057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:53.695057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:01:53.695057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:53.697387 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 271/544] 10.222.0.98 () {56 vars in 1102 bytes} [Thu Aug 18 12:01:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:53.705348 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f71241a-5b70-461e-ae60-6d428bd8d309 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:53.771248 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:53.841394 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f71241a-5b70-461e-ae60-6d428bd8d309 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:53.889984 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a9677d-471e-460b-b13a-dc616b887d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:53.890873 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9677d-471e-460b-b13a-dc616b887d64 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:53.891299 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9677d-471e-460b-b13a-dc616b887d64 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:53.891768 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9677d-471e-460b-b13a-dc616b887d64 None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:53.892610 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a9677d-471e-460b-b13a-dc616b887d64 None admin] RBAC: Adding query filter params (name=service) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:53.893063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a9677d-471e-460b-b13a-dc616b887d64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:53.894589 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a9677d-471e-460b-b13a-dc616b887d64 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:53.914844 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 274/545] 10.222.0.98 () {56 vars in 1103 bytes} [Thu Aug 18 12:01:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:53.920411 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04751727-0f5d-4a10-878a-bc9447f4bf38 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:53.956734 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ef2c19-14c9-4b84-83d3-542748c51dd2 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:53.957252 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ef2c19-14c9-4b84-83d3-542748c51dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:53.957252 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ef2c19-14c9-4b84-83d3-542748c51dd2 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:53.959057 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 275/546] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 12:01:53.964436 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de099e3-1c63-420c-9e3a-9ecf4294fbc2 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:53.964807 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de099e3-1c63-420c-9e3a-9ecf4294fbc2 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:53.965158 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de099e3-1c63-420c-9e3a-9ecf4294fbc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:54.001806 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-9de099e3-1c63-420c-9e3a-9ecf4294fbc2 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:54.003350 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-9de099e3-1c63-420c-9e3a-9ecf4294fbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:54.041977 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:54.078986 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04751727-0f5d-4a10-878a-bc9447f4bf38 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:54.127270 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 276/547] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 12:01:54.150855 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bcc930-e325-455f-aac9-62fc26c42b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:54.151551 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcc930-e325-455f-aac9-62fc26c42b2b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:54.151795 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcc930-e325-455f-aac9-62fc26c42b2b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:54.152067 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcc930-e325-455f-aac9-62fc26c42b2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:54.163936 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bcc930-e325-455f-aac9-62fc26c42b2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:54.165514 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bcc930-e325-455f-aac9-62fc26c42b2b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:54.185916 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-55bcc930-e325-455f-aac9-62fc26c42b2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:54.185916 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:54.185916 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:54.185916 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:54.185916 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:54.185916 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:54.185916 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:54.185916 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:54.185916 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:54.185916 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:54.185916 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:54.185916 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:54.185916 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:01:54.185916 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:01:54.185916 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:01:54.185916 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:01:54.186960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:54.186960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:54.186960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:54.186960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:54.186960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:54.186960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:54.186960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:54.186960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:54.186960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:54.186960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:54.186960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:54.186960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:54.186960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:54.186960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:54.186960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:54.186960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:54.186960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:01:54.187871 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:01:54.187871 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:54.187871 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:54.187871 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 272/548] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:01:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:54.192660 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de099e3-1c63-420c-9e3a-9ecf4294fbc2 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:54.244201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:54.307648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de099e3-1c63-420c-9e3a-9ecf4294fbc2 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:54.343075 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974355b-8699-437f-b4d0-cd6564d6e4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:54.343866 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974355b-8699-437f-b4d0-cd6564d6e4e2 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:54.344114 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974355b-8699-437f-b4d0-cd6564d6e4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:54.344268 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974355b-8699-437f-b4d0-cd6564d6e4e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:54.344953 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6974355b-8699-437f-b4d0-cd6564d6e4e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:54.345254 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6974355b-8699-437f-b4d0-cd6564d6e4e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:54.346675 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6974355b-8699-437f-b4d0-cd6564d6e4e2 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:54.358860 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 277/549] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:54.364054 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bcc930-e325-455f-aac9-62fc26c42b2b None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:54.417988 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974355b-8699-437f-b4d0-cd6564d6e4e2 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:54.524271 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bcc930-e325-455f-aac9-62fc26c42b2b None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:54.571877 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:54.577686 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:54.583909 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974355b-8699-437f-b4d0-cd6564d6e4e2 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:54.597169 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a852d0-5d08-44f4-a883-942157ecb444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:54.597816 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a852d0-5d08-44f4-a883-942157ecb444 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:54.598101 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a852d0-5d08-44f4-a883-942157ecb444 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:54.598377 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a852d0-5d08-44f4-a883-942157ecb444 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:54.616837 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a852d0-5d08-44f4-a883-942157ecb444 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:54.618345 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a852d0-5d08-44f4-a883-942157ecb444 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:54.638258 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-96a852d0-5d08-44f4-a883-942157ecb444 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 12:01:54.638258 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:54.638258 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:54.638258 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:54.638258 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:54.638258 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:54.638258 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:54.638258 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:54.638258 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:54.638258 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:54.638258 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:54.638258 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:54.638258 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 12:01:54.638258 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 12:01:54.638258 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 12:01:54.638258 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 12:01:54.639234 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:54.639234 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:54.639234 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:01:54.639234 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:01:54.639234 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:01:54.639234 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:01:54.639234 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:54.639234 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:54.639234 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:54.639234 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:54.639234 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:54.639234 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:54.639234 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:54.639234 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:54.639234 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:54.639234 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:54.639234 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:54.639234 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:54.640186 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:54.640186 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:54.640186 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 12:01:54.640186 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 12:01:54.640186 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 12:01:54.640186 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 12:01:54.640186 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 12:01:54.640186 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 12:01:54.640186 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 12:01:54.640186 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:54.640733 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3b42a-0ea9-4da3-8c15-9a5ce952af31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:54.641321 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3b42a-0ea9-4da3-8c15-9a5ce952af31 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:54.641321 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3b42a-0ea9-4da3-8c15-9a5ce952af31 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:54.641482 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3b42a-0ea9-4da3-8c15-9a5ce952af31 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:54.642603 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 273/550] 10.222.0.98 () {56 vars in 1093 bytes} [Thu Aug 18 12:01:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:54.647357 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a852d0-5d08-44f4-a883-942157ecb444 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:54.663720 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd3b42a-0ea9-4da3-8c15-9a5ce952af31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:54.674726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-6cd3b42a-0ea9-4da3-8c15-9a5ce952af31 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:01:54.674726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:54.674726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:54.674726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:54.674726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:54.674726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:54.674726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:54.674726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:54.674726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:54.674726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:54.674726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:54.674726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:54.674726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:54.674726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:54.674726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:54.674726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:54.675917 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:54.675917 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:54.675917 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:54.675917 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:54.675917 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:54.675917 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:54.675917 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:54.675917 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:54.675917 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:54.675917 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:54.675917 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:54.675917 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:54.675917 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:54.675917 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:54.675917 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:54.675917 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:54.676774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:54.676774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:54.676774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:54.676774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:54.676774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:54.676774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:54.676774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:54.676774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:54.676774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:54.676774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:54.676774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:01:54.676774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:54.676774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 278/551] 10.222.0.98 () {56 vars in 1102 bytes} [Thu Aug 18 12:01:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:01:54.686053 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3b42a-0ea9-4da3-8c15-9a5ce952af31 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:54.838113 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3b42a-0ea9-4da3-8c15-9a5ce952af31 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:54.874523 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8415eed-e613-418f-8812-29e0497ab536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:54.875361 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8415eed-e613-418f-8812-29e0497ab536 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:54.875729 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8415eed-e613-418f-8812-29e0497ab536 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:54.876009 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8415eed-e613-418f-8812-29e0497ab536 None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:54.876679 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8415eed-e613-418f-8812-29e0497ab536 None admin] RBAC: Adding query filter params (name=service) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:54.876989 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8415eed-e613-418f-8812-29e0497ab536 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:54.878421 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8415eed-e613-418f-8812-29e0497ab536 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:54.885322 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a852d0-5d08-44f4-a883-942157ecb444 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:54.887274 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 279/552] 10.222.0.98 () {56 vars in 1103 bytes} [Thu Aug 18 12:01:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:54.893440 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8415eed-e613-418f-8812-29e0497ab536 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:54.919168 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:54.954828 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efbf50a-3b8a-4817-bbee-8cf4d0563ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:54.955686 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efbf50a-3b8a-4817-bbee-8cf4d0563ee2 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:54.955982 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efbf50a-3b8a-4817-bbee-8cf4d0563ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:54.956261 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efbf50a-3b8a-4817-bbee-8cf4d0563ee2 None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:54.957006 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efbf50a-3b8a-4817-bbee-8cf4d0563ee2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:54.957320 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efbf50a-3b8a-4817-bbee-8cf4d0563ee2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:54.958757 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efbf50a-3b8a-4817-bbee-8cf4d0563ee2 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:54.979558 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 274/553] 10.222.0.98 () {56 vars in 1133 bytes} [Thu Aug 18 12:01:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:54.985162 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efbf50a-3b8a-4817-bbee-8cf4d0563ee2 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:55.064938 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8415eed-e613-418f-8812-29e0497ab536 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:55.067852 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:55.104570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eccb7e-1a87-41e2-b621-c8484c461636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:55.105129 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eccb7e-1a87-41e2-b621-c8484c461636 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:55.105416 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eccb7e-1a87-41e2-b621-c8484c461636 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:55.105698 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eccb7e-1a87-41e2-b621-c8484c461636 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:55.121563 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eccb7e-1a87-41e2-b621-c8484c461636 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:55.123043 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eccb7e-1a87-41e2-b621-c8484c461636 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:55.137869 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-69eccb7e-1a87-41e2-b621-c8484c461636 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:55.137869 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:55.137869 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:55.137869 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:55.137869 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:55.137869 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:55.137869 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:55.137869 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:55.137869 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:55.137869 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:55.137869 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:55.137869 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:55.137869 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:01:55.137869 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:01:55.137869 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:01:55.137869 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:01:55.138838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:55.138838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:55.138838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:55.138838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:55.138838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:55.138838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:55.138838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:55.138838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:55.138838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:55.138838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:55.138838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:55.138838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:55.138838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:55.138838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:55.138838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:55.138838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:55.138838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:01:55.139746 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:01:55.139746 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:55.139746 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:55.139746 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 280/554] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:01:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:55.145184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eccb7e-1a87-41e2-b621-c8484c461636 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:55.176036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efbf50a-3b8a-4817-bbee-8cf4d0563ee2 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:55.222657 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:55.236314 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cb9381-ce36-4353-9267-4cb8214bc74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:55.240283 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb9381-ce36-4353-9267-4cb8214bc74b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:55.240283 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb9381-ce36-4353-9267-4cb8214bc74b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:55.240283 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb9381-ce36-4353-9267-4cb8214bc74b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:55.250516 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cb9381-ce36-4353-9267-4cb8214bc74b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:55.251901 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cb9381-ce36-4353-9267-4cb8214bc74b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:55.257134 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eccb7e-1a87-41e2-b621-c8484c461636 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:55.281643 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-21cb9381-ce36-4353-9267-4cb8214bc74b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:55.281643 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:55.281643 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:55.281643 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:55.281643 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:55.281643 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:55.281643 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:55.281643 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:55.281643 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:55.281643 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:55.281643 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:55.281643 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:55.281643 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:01:55.281643 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:01:55.281643 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:01:55.281643 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:01:55.282625 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:55.282625 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:55.282625 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:55.282625 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:55.282625 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:55.282625 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:55.282625 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:55.282625 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:55.282625 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:55.282625 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:55.282625 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:55.282625 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:55.282625 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:55.282625 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:55.282625 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:55.282625 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:55.282625 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:01:55.284561 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:01:55.284561 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:55.284561 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:55.284561 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 275/555] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:01:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:55.285882 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f78ae8-2fb4-4bdb-9322-93cf2fa269eb None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:55.286167 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f78ae8-2fb4-4bdb-9322-93cf2fa269eb None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:55.286439 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f78ae8-2fb4-4bdb-9322-93cf2fa269eb None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:55.288298 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 276/556] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:55.289992 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f78ae8-2fb4-4bdb-9322-93cf2fa269eb None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:55.300213 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb74870-023b-4c40-b1fe-e74fe52e6c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:55.301125 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb74870-023b-4c40-b1fe-e74fe52e6c78 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:55.301615 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb74870-023b-4c40-b1fe-e74fe52e6c78 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:55.302097 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb74870-023b-4c40-b1fe-e74fe52e6c78 None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:55.302987 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb74870-023b-4c40-b1fe-e74fe52e6c78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:55.303543 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb74870-023b-4c40-b1fe-e74fe52e6c78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:55.305130 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb74870-023b-4c40-b1fe-e74fe52e6c78 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:55.318219 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 281/557] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:01:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:01:55.324089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242b2ec-faeb-45af-a212-26b86aeb321c None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:55.324644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242b2ec-faeb-45af-a212-26b86aeb321c None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:55.325156 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242b2ec-faeb-45af-a212-26b86aeb321c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:55.344416 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-9242b2ec-faeb-45af-a212-26b86aeb321c None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:55.345997 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-9242b2ec-faeb-45af-a212-26b86aeb321c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:55.391897 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:55.398060 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f78ae8-2fb4-4bdb-9322-93cf2fa269eb None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:55.419907 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 282/558] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:55.421085 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242b2ec-faeb-45af-a212-26b86aeb321c None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:55.449362 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2defdd-9a21-498f-8c3a-6dcc38c23001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:55.450027 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2defdd-9a21-498f-8c3a-6dcc38c23001 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:55.450308 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2defdd-9a21-498f-8c3a-6dcc38c23001 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:55.450632 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2defdd-9a21-498f-8c3a-6dcc38c23001 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:55.451290 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2defdd-9a21-498f-8c3a-6dcc38c23001 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:55.451621 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2defdd-9a21-498f-8c3a-6dcc38c23001 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:55.453660 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2defdd-9a21-498f-8c3a-6dcc38c23001 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:55.483948 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 277/559] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:01:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:55.489078 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2defdd-9a21-498f-8c3a-6dcc38c23001 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:55.491220 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:55.547476 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242b2ec-faeb-45af-a212-26b86aeb321c None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:55.586730 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9efa9b7-4249-4c00-a9df-c4a9edf671b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:55.587907 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9efa9b7-4249-4c00-a9df-c4a9edf671b7 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:55.592857 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9efa9b7-4249-4c00-a9df-c4a9edf671b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:55.592857 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9efa9b7-4249-4c00-a9df-c4a9edf671b7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:55.598045 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9efa9b7-4249-4c00-a9df-c4a9edf671b7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:55.599917 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9efa9b7-4249-4c00-a9df-c4a9edf671b7 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:55.612035 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-c9efa9b7-4249-4c00-a9df-c4a9edf671b7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 18 12:01:55.612035 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:55.612035 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:55.612035 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:55.612035 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:55.612035 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:55.612035 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:55.612035 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:55.612035 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:55.612035 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:55.612035 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:55.612035 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:55.612035 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 12:01:55.612035 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 12:01:55.612035 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 12:01:55.612035 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 12:01:55.613029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:55.613029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:55.613029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:01:55.613029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:01:55.613029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:01:55.613029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:01:55.613029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:55.613029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:55.613029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:55.613029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:55.613029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:55.613029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:55.613029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:55.613029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:55.613029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:55.613029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:55.613029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:55.613029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:55.613984 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:55.613984 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:55.613984 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 12:01:55.613984 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 12:01:55.613984 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 12:01:55.613984 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 12:01:55.613984 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 12:01:55.613984 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 12:01:55.613984 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 18 12:01:55.613984 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:55.617952 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 283/560] 10.222.0.98 () {56 vars in 1096 bytes} [Thu Aug 18 12:01:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:55.618065 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2defdd-9a21-498f-8c3a-6dcc38c23001 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:55.619166 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02616244-176a-4b0c-90d2-4c46f7c4b30a None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:55.619490 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02616244-176a-4b0c-90d2-4c46f7c4b30a None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:55.619775 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02616244-176a-4b0c-90d2-4c46f7c4b30a None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:55.621185 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 284/561] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:55.622053 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0344a6-bb30-4c8a-b755-93dd2099e63a None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:55.622402 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0344a6-bb30-4c8a-b755-93dd2099e63a None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:55.622682 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0344a6-bb30-4c8a-b755-93dd2099e63a None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:55.625444 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 285/562] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:55.625444 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0344a6-bb30-4c8a-b755-93dd2099e63a None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:55.653933 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e77cecb-3609-482a-a714-ddb261c502db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:55.654488 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77cecb-3609-482a-a714-ddb261c502db None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:55.654769 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77cecb-3609-482a-a714-ddb261c502db None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:55.655035 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77cecb-3609-482a-a714-ddb261c502db None admin] PATH_INFO: `/v3/projects/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:55.662701 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e77cecb-3609-482a-a714-ddb261c502db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:55.664122 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e77cecb-3609-482a-a714-ddb261c502db None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:55.676443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-6e77cecb-3609-482a-a714-ddb261c502db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:01:55.676443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:55.676443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:55.676443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:55.676443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:55.676443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:55.676443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:55.676443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:55.676443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:55.676443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:55.676443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:55.676443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:55.676443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 12:01:55.676443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 12:01:55.676443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 12:01:55.677545 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 12:01:55.677545 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:55.677545 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:55.677545 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:55.677545 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:55.677545 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:55.677545 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:55.677545 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:55.677545 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:55.677545 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:55.677545 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:55.677545 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:55.677545 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:55.677545 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:55.677545 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:55.677545 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:55.677545 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:55.678700 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 12:01:55.678700 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 12:01:55.678700 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:01:55.678700 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:01:55.678700 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:01:55.678700 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:01:55.678700 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:01:55.678700 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:55.678700 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 278/563] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:01:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:01:55.679182 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7ff61-c5b6-4e8b-9394-283fb2729ef3 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:55.679364 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7ff61-c5b6-4e8b-9394-283fb2729ef3 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:55.679684 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7ff61-c5b6-4e8b-9394-283fb2729ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:55.700971 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-cfb7ff61-c5b6-4e8b-9394-283fb2729ef3 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:55.702195 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-cfb7ff61-c5b6-4e8b-9394-283fb2729ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:55.713379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:55.749176 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0344a6-bb30-4c8a-b755-93dd2099e63a None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:55.773734 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 279/564] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:55.774976 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec20b3-ee44-4aee-bd3e-e0d164636608 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:55.775267 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec20b3-ee44-4aee-bd3e-e0d164636608 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:55.775573 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec20b3-ee44-4aee-bd3e-e0d164636608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:55.783004 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:55.794728 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed197b2e-7198-4c7b-b981-be340eef2776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:55.795454 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed197b2e-7198-4c7b-b981-be340eef2776 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:55.795796 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed197b2e-7198-4c7b-b981-be340eef2776 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:55.796140 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed197b2e-7198-4c7b-b981-be340eef2776 None admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:55.796989 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed197b2e-7198-4c7b-b981-be340eef2776 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:55.797358 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed197b2e-7198-4c7b-b981-be340eef2776 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:55.798220 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-23ec20b3-ee44-4aee-bd3e-e0d164636608 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:55.798927 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed197b2e-7198-4c7b-b981-be340eef2776 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:55.799604 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-23ec20b3-ee44-4aee-bd3e-e0d164636608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:55.822314 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 286/565] 10.222.0.98 () {56 vars in 1135 bytes} [Thu Aug 18 12:01:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:55.824353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed197b2e-7198-4c7b-b981-be340eef2776 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:55.874004 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 280/566] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:55.875707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ec20b3-ee44-4aee-bd3e-e0d164636608 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:55.945222 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed197b2e-7198-4c7b-b981-be340eef2776 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:55.977814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:55.993022 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6b513-31b9-4865-b380-e0a01281a09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:55.994143 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6b513-31b9-4865-b380-e0a01281a09e None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:55.994961 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6b513-31b9-4865-b380-e0a01281a09e None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:55.995715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6b513-31b9-4865-b380-e0a01281a09e None admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:55.996750 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e6b513-31b9-4865-b380-e0a01281a09e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:55.997500 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e6b513-31b9-4865-b380-e0a01281a09e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:55.999648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e6b513-31b9-4865-b380-e0a01281a09e None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:56.003711 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ec20b3-ee44-4aee-bd3e-e0d164636608 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:56.019829 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 287/567] 10.222.0.98 () {56 vars in 1148 bytes} [Thu Aug 18 12:01:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:56.019829 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6b513-31b9-4865-b380-e0a01281a09e None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:56.055470 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a26b65-55cd-4559-b173-333129d12da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:56.056479 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a26b65-55cd-4559-b173-333129d12da4 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:56.056918 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a26b65-55cd-4559-b173-333129d12da4 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:56.057344 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a26b65-55cd-4559-b173-333129d12da4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:56.067317 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:56.073766 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a26b65-55cd-4559-b173-333129d12da4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:56.079289 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-08a26b65-55cd-4559-b173-333129d12da4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:01:56.079289 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:56.079289 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:56.079289 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:56.079289 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:56.079289 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:56.079289 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:56.079289 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:56.079289 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:56.079289 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:56.079289 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:56.079289 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:56.079289 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:56.079289 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:56.079289 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:56.079289 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:56.081875 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:56.081875 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:56.081875 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:56.081875 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:56.081875 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:56.081875 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:56.081875 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:56.081875 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:56.081875 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:56.081875 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:56.081875 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:56.081875 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:56.081875 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:56.081875 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:56.081875 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:56.081875 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:56.082707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:56.082707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:56.082707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:56.082707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:56.082707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:56.082707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:56.082707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:56.082707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:56.082707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:56.082707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:56.082707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:01:56.082707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:56.082707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 281/568] 10.222.0.98 () {56 vars in 1102 bytes} [Thu Aug 18 12:01:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:56.082707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a26b65-55cd-4559-b173-333129d12da4 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:56.146649 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6b513-31b9-4865-b380-e0a01281a09e None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:56.181237 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a013d43c-6113-4c1d-9c86-a688913f11a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:56.181843 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013d43c-6113-4c1d-9c86-a688913f11a6 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:56.182131 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013d43c-6113-4c1d-9c86-a688913f11a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:56.182407 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013d43c-6113-4c1d-9c86-a688913f11a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:56.190639 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a26b65-55cd-4559-b173-333129d12da4 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:56.224995 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee0eca9-2016-4f09-b09d-4fd09a6996b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:56.225660 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0eca9-2016-4f09-b09d-4fd09a6996b5 None admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:56.225933 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0eca9-2016-4f09-b09d-4fd09a6996b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:56.226223 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0eca9-2016-4f09-b09d-4fd09a6996b5 None admin] PATH_INFO: `/v3/projects/6f676a5169464d83af4267db25025728/users/171c72a6b29e418bbb4ea5c6faffe3b3/roles/563e3ed5c6164aae966b64cc302b114a` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:56.228924 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:56.241779 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee0eca9-2016-4f09-b09d-4fd09a6996b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f676a5169464d83af4267db25025728, user_id=171c72a6b29e418bbb4ea5c6faffe3b3, role_id=563e3ed5c6164aae966b64cc302b114a)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:56.243219 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee0eca9-2016-4f09-b09d-4fd09a6996b5 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:56.251001 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a013d43c-6113-4c1d-9c86-a688913f11a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:56.252464 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a013d43c-6113-4c1d-9c86-a688913f11a6 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:56.291196 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:56.315211 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 282/569] 10.222.0.98 () {58 vars in 1439 bytes} [Thu Aug 18 12:01:56 2022] PUT /identity/v3/projects/6f676a5169464d83af4267db25025728/users/171c72a6b29e418bbb4ea5c6faffe3b3/roles/563e3ed5c6164aae966b64cc302b114a => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:01:56.316346 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee0eca9-2016-4f09-b09d-4fd09a6996b5 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:56.354658 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 288/570] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:01:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:01:56.355401 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a013d43c-6113-4c1d-9c86-a688913f11a6 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:56.446898 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee0eca9-2016-4f09-b09d-4fd09a6996b5 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:56.486289 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7356f7-3f63-41d1-9ae2-7574c52b5156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:56.486849 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7356f7-3f63-41d1-9ae2-7574c52b5156 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:56.487131 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7356f7-3f63-41d1-9ae2-7574c52b5156 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:56.487409 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7356f7-3f63-41d1-9ae2-7574c52b5156 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:56.493993 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a013d43c-6113-4c1d-9c86-a688913f11a6 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:56.498096 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7356f7-3f63-41d1-9ae2-7574c52b5156 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:56.503607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-2c7356f7-3f63-41d1-9ae2-7574c52b5156 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:01:56.503607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:56.503607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:56.503607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:56.503607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:56.503607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:56.503607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:56.503607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:56.503607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:56.503607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:56.503607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:56.503607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:56.503607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:56.503607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:56.503607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:56.503607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:56.504775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:56.504775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:56.504775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:56.504775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:56.504775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:56.504775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:56.504775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:56.504775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:56.504775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:56.504775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:56.504775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:56.504775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:56.504775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:56.504775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:56.504775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:56.504775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:56.505785 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:56.505785 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:56.505785 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:56.505785 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:56.505785 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:56.505785 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:56.505785 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:56.505785 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:56.505785 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:56.505785 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:56.505785 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:01:56.505785 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:56.505785 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 283/571] 10.222.0.98 () {56 vars in 1102 bytes} [Thu Aug 18 12:01:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:56.505785 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7356f7-3f63-41d1-9ae2-7574c52b5156 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:56.534794 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa9fc7d-b06e-4f50-ac36-bf1478450616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:56.535407 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9fc7d-b06e-4f50-ac36-bf1478450616 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:56.535723 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9fc7d-b06e-4f50-ac36-bf1478450616 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:56.536004 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9fc7d-b06e-4f50-ac36-bf1478450616 None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:56.536673 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa9fc7d-b06e-4f50-ac36-bf1478450616 None admin] RBAC: Adding query filter params (name=service) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:56.536984 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa9fc7d-b06e-4f50-ac36-bf1478450616 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:56.538371 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa9fc7d-b06e-4f50-ac36-bf1478450616 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:56.547382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 289/572] 10.222.0.98 () {56 vars in 1103 bytes} [Thu Aug 18 12:01:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:56.548158 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa9fc7d-b06e-4f50-ac36-bf1478450616 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:56.550844 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:56.563916 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:56.620317 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7356f7-3f63-41d1-9ae2-7574c52b5156 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:56.658608 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0c31d-a574-40fd-80f6-c8b89f006038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:56.659322 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0c31d-a574-40fd-80f6-c8b89f006038 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:56.659694 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0c31d-a574-40fd-80f6-c8b89f006038 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:56.660034 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0c31d-a574-40fd-80f6-c8b89f006038 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:56.661106 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa9fc7d-b06e-4f50-ac36-bf1478450616 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:56.674362 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df0c31d-a574-40fd-80f6-c8b89f006038 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:56.679810 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-7df0c31d-a574-40fd-80f6-c8b89f006038 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:01:56.679810 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:56.679810 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:56.679810 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:56.679810 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:56.679810 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:56.679810 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:56.679810 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:56.679810 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:56.679810 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:56.679810 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:56.679810 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:56.679810 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:56.679810 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:56.679810 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:56.679810 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:56.681027 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:56.681027 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:56.681027 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:56.681027 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:56.681027 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:56.681027 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:56.681027 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:56.681027 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:56.681027 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:56.681027 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:56.681027 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:56.681027 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:56.681027 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:56.681027 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:56.681027 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:56.681027 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:56.682065 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:56.682065 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:56.682065 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:56.682065 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:56.682065 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:56.682065 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:56.682065 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:56.682065 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:56.682065 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:56.682065 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:56.682065 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:01:56.682065 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:56.682789 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 284/573] 10.222.0.98 () {56 vars in 1102 bytes} [Thu Aug 18 12:01:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:56.683603 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0c31d-a574-40fd-80f6-c8b89f006038 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:56.704462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64901885-7384-4619-bc29-3bb54b9bbe65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:56.705105 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64901885-7384-4619-bc29-3bb54b9bbe65 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:56.705389 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64901885-7384-4619-bc29-3bb54b9bbe65 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:56.705666 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64901885-7384-4619-bc29-3bb54b9bbe65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:56.717359 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64901885-7384-4619-bc29-3bb54b9bbe65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:56.719353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64901885-7384-4619-bc29-3bb54b9bbe65 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:56.735995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-64901885-7384-4619-bc29-3bb54b9bbe65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:56.735995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:56.735995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:56.735995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:56.735995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:56.735995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:56.735995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:56.735995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:56.735995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:56.735995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:56.735995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:56.735995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:56.735995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:01:56.735995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:01:56.735995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:01:56.735995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:01:56.737227 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:56.737227 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:56.737227 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:56.737227 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:56.737227 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:56.737227 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:56.737227 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:56.737227 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:56.737227 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:56.737227 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:56.737227 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:56.737227 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:56.737227 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:56.737227 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:56.737227 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:56.737227 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:56.737227 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:01:56.738142 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:01:56.738142 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:56.738142 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:56.738142 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 290/574] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:01:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:56.738142 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64901885-7384-4619-bc29-3bb54b9bbe65 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:56.805370 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:56.805568 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0c31d-a574-40fd-80f6-c8b89f006038 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:56.809139 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:56.843014 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab07901-b874-4b8e-8598-8ecac5bd7b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:56.844298 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab07901-b874-4b8e-8598-8ecac5bd7b83 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:56.844519 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab07901-b874-4b8e-8598-8ecac5bd7b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:56.844795 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab07901-b874-4b8e-8598-8ecac5bd7b83 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:56.845467 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab07901-b874-4b8e-8598-8ecac5bd7b83 None admin] RBAC: Adding query filter params (name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:56.845771 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab07901-b874-4b8e-8598-8ecac5bd7b83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:56.847137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab07901-b874-4b8e-8598-8ecac5bd7b83 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:56.855850 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 285/575] 10.222.0.98 () {56 vars in 1103 bytes} [Thu Aug 18 12:01:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:56.856680 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab07901-b874-4b8e-8598-8ecac5bd7b83 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:56.863031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64901885-7384-4619-bc29-3bb54b9bbe65 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:56.901328 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2948d6b-cbf1-4cdb-913f-cc199f2a4bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:56.902031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2948d6b-cbf1-4cdb-913f-cc199f2a4bf1 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:56.902216 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2948d6b-cbf1-4cdb-913f-cc199f2a4bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:56.902482 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2948d6b-cbf1-4cdb-913f-cc199f2a4bf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:56.914431 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2948d6b-cbf1-4cdb-913f-cc199f2a4bf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:56.915901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2948d6b-cbf1-4cdb-913f-cc199f2a4bf1 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:56.930846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-e2948d6b-cbf1-4cdb-913f-cc199f2a4bf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:56.930846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:56.930846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:56.930846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:56.930846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:56.930846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:56.930846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:56.930846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:56.930846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:56.930846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:56.930846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:56.930846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:56.930846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:01:56.930846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:01:56.930846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:01:56.930846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:01:56.931933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:56.931933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:56.931933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:56.931933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:56.931933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:56.931933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:56.931933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:56.931933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:56.931933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:56.931933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:56.931933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:56.931933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:56.931933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:56.931933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:56.931933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:56.931933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:56.931933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:01:56.932805 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:01:56.932805 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:56.932805 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:56.932805 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 291/576] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:01:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:56.932805 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2948d6b-cbf1-4cdb-913f-cc199f2a4bf1 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:56.972560 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab07901-b874-4b8e-8598-8ecac5bd7b83 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:57.009235 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed829ad8-4715-4e5c-a352-29fb1190c360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:57.010070 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed829ad8-4715-4e5c-a352-29fb1190c360 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:57.010527 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed829ad8-4715-4e5c-a352-29fb1190c360 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:57.010963 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed829ad8-4715-4e5c-a352-29fb1190c360 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:57.011656 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed829ad8-4715-4e5c-a352-29fb1190c360 None admin] RBAC: Adding query filter params (name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:57.011966 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed829ad8-4715-4e5c-a352-29fb1190c360 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:57.013330 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed829ad8-4715-4e5c-a352-29fb1190c360 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:57.026667 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 286/577] 10.222.0.98 () {56 vars in 1103 bytes} [Thu Aug 18 12:01:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:57.026667 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed829ad8-4715-4e5c-a352-29fb1190c360 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:57.052448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:57.052613 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2948d6b-cbf1-4cdb-913f-cc199f2a4bf1 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:57.060475 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:57.089517 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c33e0-9259-490c-88d2-bf9c6696e0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:57.090507 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c33e0-9259-490c-88d2-bf9c6696e0ed None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:57.090774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c33e0-9259-490c-88d2-bf9c6696e0ed None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:57.091059 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c33e0-9259-490c-88d2-bf9c6696e0ed None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:57.091739 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7c33e0-9259-490c-88d2-bf9c6696e0ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:57.092050 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7c33e0-9259-490c-88d2-bf9c6696e0ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:57.093416 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7c33e0-9259-490c-88d2-bf9c6696e0ed None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:57.105859 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 292/578] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:01:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:57.106678 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c33e0-9259-490c-88d2-bf9c6696e0ed None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:57.135257 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed829ad8-4715-4e5c-a352-29fb1190c360 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:57.175836 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb6eff-9bc0-498b-9aaa-9dce6a24934b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:57.176652 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb6eff-9bc0-498b-9aaa-9dce6a24934b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:57.176934 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb6eff-9bc0-498b-9aaa-9dce6a24934b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:57.177246 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb6eff-9bc0-498b-9aaa-9dce6a24934b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:57.188592 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cb6eff-9bc0-498b-9aaa-9dce6a24934b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:57.190112 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cb6eff-9bc0-498b-9aaa-9dce6a24934b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:57.205158 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-35cb6eff-9bc0-498b-9aaa-9dce6a24934b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:57.205158 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:57.205158 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:57.205158 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:57.205158 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:57.205158 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:57.205158 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:57.205158 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:57.205158 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:57.205158 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:57.205158 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:57.205158 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:57.205158 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:01:57.205158 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:01:57.205158 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:01:57.205158 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:01:57.207078 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:57.207078 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:57.207078 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:57.207078 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:57.207078 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:57.207078 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:57.207078 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:57.207078 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:57.207078 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:57.207078 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:57.207078 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:57.207078 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:57.207078 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:57.207078 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:57.207078 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:57.207078 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:57.207078 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:01:57.208095 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:01:57.208095 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:57.208095 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:57.208095 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 287/579] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:01:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:57.208095 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb6eff-9bc0-498b-9aaa-9dce6a24934b None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:57.228202 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c33e0-9259-490c-88d2-bf9c6696e0ed None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:57.271451 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81a382-b36d-4f95-b230-32ce62cdb910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:57.271973 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81a382-b36d-4f95-b230-32ce62cdb910 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:57.272248 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81a382-b36d-4f95-b230-32ce62cdb910 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:57.272556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81a382-b36d-4f95-b230-32ce62cdb910 None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:57.273158 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a81a382-b36d-4f95-b230-32ce62cdb910 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:57.273842 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a81a382-b36d-4f95-b230-32ce62cdb910 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:57.275079 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a81a382-b36d-4f95-b230-32ce62cdb910 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:57.288773 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 293/580] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:01:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:57.288773 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81a382-b36d-4f95-b230-32ce62cdb910 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:57.298736 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:57.320332 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb6eff-9bc0-498b-9aaa-9dce6a24934b None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:57.321540 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:57.355560 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a86c1c-6ece-49ca-85b4-d6c6964f2e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:57.356313 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a86c1c-6ece-49ca-85b4-d6c6964f2e3b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:57.356657 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a86c1c-6ece-49ca-85b4-d6c6964f2e3b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:57.357006 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a86c1c-6ece-49ca-85b4-d6c6964f2e3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:57.369606 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a86c1c-6ece-49ca-85b4-d6c6964f2e3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:57.371177 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a86c1c-6ece-49ca-85b4-d6c6964f2e3b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:57.387256 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-b4a86c1c-6ece-49ca-85b4-d6c6964f2e3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:57.387256 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:57.387256 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:57.387256 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:57.387256 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:57.387256 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:57.387256 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:57.387256 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:57.387256 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:57.387256 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:57.387256 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:57.387256 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:57.387256 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:01:57.387256 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:01:57.387256 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:01:57.387256 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:01:57.388571 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:57.388571 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:57.388571 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:57.388571 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:57.388571 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:57.388571 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:57.388571 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:57.388571 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:57.388571 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:57.388571 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:57.388571 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:57.388571 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:57.388571 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:57.388571 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:57.388571 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:57.388571 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:57.388571 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:01:57.389680 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:01:57.389680 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:57.389680 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:57.394658 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 288/581] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:01:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:57.395650 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a86c1c-6ece-49ca-85b4-d6c6964f2e3b None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:57.404219 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81a382-b36d-4f95-b230-32ce62cdb910 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:57.442420 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592eb08-da8d-4155-8d2b-550df5ca7b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:57.443206 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592eb08-da8d-4155-8d2b-550df5ca7b7a None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:57.443745 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592eb08-da8d-4155-8d2b-550df5ca7b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:57.444094 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592eb08-da8d-4155-8d2b-550df5ca7b7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:57.451290 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8592eb08-da8d-4155-8d2b-550df5ca7b7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:57.452785 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8592eb08-da8d-4155-8d2b-550df5ca7b7a None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:57.465802 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-8592eb08-da8d-4155-8d2b-550df5ca7b7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:01:57.465802 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:57.465802 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:57.465802 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:57.465802 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:57.465802 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:57.465802 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:57.465802 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:57.465802 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:57.465802 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:57.465802 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:57.465802 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:57.465802 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 12:01:57.465802 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 12:01:57.465802 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 12:01:57.466769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 12:01:57.466769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:57.466769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:57.466769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:57.466769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:57.466769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:57.466769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:57.466769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:57.466769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:57.466769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:57.466769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:57.466769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:57.466769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:57.466769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:57.466769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:57.466769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:57.466769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:57.468095 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 12:01:57.468095 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 12:01:57.468095 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:01:57.468095 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:01:57.468095 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:01:57.468095 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:01:57.468095 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:01:57.468095 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:57.468095 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 294/582] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:01:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:57.468095 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592eb08-da8d-4155-8d2b-550df5ca7b7a None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:57.506604 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a86c1c-6ece-49ca-85b4-d6c6964f2e3b None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:57.538616 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:57.545490 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16165997-2aee-4f38-be63-6c5abd165612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:57.546177 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16165997-2aee-4f38-be63-6c5abd165612 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:57.546465 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16165997-2aee-4f38-be63-6c5abd165612 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:57.546740 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16165997-2aee-4f38-be63-6c5abd165612 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:57.548228 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16165997-2aee-4f38-be63-6c5abd165612 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:57.555479 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16165997-2aee-4f38-be63-6c5abd165612 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:57.555479 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16165997-2aee-4f38-be63-6c5abd165612 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:57.561498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 289/583] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:01:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:57.562746 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16165997-2aee-4f38-be63-6c5abd165612 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:57.568376 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:57.585100 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592eb08-da8d-4155-8d2b-550df5ca7b7a None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:57.624127 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d76feef-f10c-4b87-9c07-c39f58670806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:57.624757 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76feef-f10c-4b87-9c07-c39f58670806 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:57.625068 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76feef-f10c-4b87-9c07-c39f58670806 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:57.625347 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76feef-f10c-4b87-9c07-c39f58670806 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:57.633369 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d76feef-f10c-4b87-9c07-c39f58670806 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:57.634806 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d76feef-f10c-4b87-9c07-c39f58670806 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:57.647225 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-6d76feef-f10c-4b87-9c07-c39f58670806 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 12:01:57.647225 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:57.647225 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:57.647225 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:57.647225 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:57.647225 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:57.647225 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:57.647225 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:57.647225 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:57.647225 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:57.647225 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:57.647225 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:57.647225 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 12:01:57.647225 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 12:01:57.647225 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 12:01:57.647225 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 12:01:57.648703 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:57.648703 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:57.648703 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:01:57.648703 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:01:57.648703 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:01:57.648703 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:01:57.648703 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:57.648703 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:57.648703 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:57.648703 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:57.648703 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:57.648703 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:57.648703 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:57.648703 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:57.648703 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:57.648703 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:57.648703 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:57.648703 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:57.649685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:57.649685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:57.649685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 12:01:57.649685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 12:01:57.649685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 12:01:57.649685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 12:01:57.649685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 12:01:57.649685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 12:01:57.649685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 12:01:57.649685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:57.649685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 295/584] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:01:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:57.650278 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d76feef-f10c-4b87-9c07-c39f58670806 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:57.677880 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16165997-2aee-4f38-be63-6c5abd165612 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:57.720647 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10afdc-921e-4b54-bdaa-e1f24d73a6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:57.721258 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10afdc-921e-4b54-bdaa-e1f24d73a6c2 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:57.721540 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10afdc-921e-4b54-bdaa-e1f24d73a6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:57.721813 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10afdc-921e-4b54-bdaa-e1f24d73a6c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:57.722461 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da10afdc-921e-4b54-bdaa-e1f24d73a6c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:57.722767 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da10afdc-921e-4b54-bdaa-e1f24d73a6c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:57.724167 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da10afdc-921e-4b54-bdaa-e1f24d73a6c2 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:57.739617 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 290/585] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:01:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:57.740344 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10afdc-921e-4b54-bdaa-e1f24d73a6c2 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:57.773711 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d76feef-f10c-4b87-9c07-c39f58670806 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:57.799316 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:57.813477 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e042d-5ca1-4115-ba98-27c53edc3eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:57.814092 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e042d-5ca1-4115-ba98-27c53edc3eb4 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:57.814378 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e042d-5ca1-4115-ba98-27c53edc3eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:57.814660 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e042d-5ca1-4115-ba98-27c53edc3eb4 None admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:57.815352 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83e042d-5ca1-4115-ba98-27c53edc3eb4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:57.815691 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83e042d-5ca1-4115-ba98-27c53edc3eb4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:57.817124 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83e042d-5ca1-4115-ba98-27c53edc3eb4 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:57.829992 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 296/586] 10.222.0.98 () {56 vars in 1148 bytes} [Thu Aug 18 12:01:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:57.830822 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e042d-5ca1-4115-ba98-27c53edc3eb4 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:57.840986 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:57.854787 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10afdc-921e-4b54-bdaa-e1f24d73a6c2 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:57.891272 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a5ddc0-8781-4531-ad48-f9f8055a18ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:57.892019 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a5ddc0-8781-4531-ad48-f9f8055a18ca None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:57.892369 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a5ddc0-8781-4531-ad48-f9f8055a18ca None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:57.892716 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a5ddc0-8781-4531-ad48-f9f8055a18ca None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:57.900949 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a5ddc0-8781-4531-ad48-f9f8055a18ca None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:57.902483 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a5ddc0-8781-4531-ad48-f9f8055a18ca None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:57.915689 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-33a5ddc0-8781-4531-ad48-f9f8055a18ca None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 18 12:01:57.915689 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:57.915689 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:57.915689 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:57.915689 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:57.915689 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:57.915689 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:57.915689 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:57.915689 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:57.915689 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:57.915689 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:57.915689 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:57.915689 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 12:01:57.915689 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 12:01:57.915689 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 12:01:57.915689 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 12:01:57.916884 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:57.916884 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:57.916884 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:01:57.916884 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:01:57.916884 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:01:57.916884 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:01:57.916884 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:57.916884 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:57.916884 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:57.916884 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:57.916884 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:57.916884 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:57.916884 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:57.916884 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:57.916884 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:57.916884 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:57.916884 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:57.916884 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:57.923006 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:57.923006 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:57.923006 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 12:01:57.923006 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 12:01:57.923006 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 12:01:57.923006 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 12:01:57.923006 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 12:01:57.923006 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 12:01:57.923006 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 18 12:01:57.923006 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:57.923006 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 291/587] 10.222.0.98 () {56 vars in 1102 bytes} [Thu Aug 18 12:01:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:57.923006 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a5ddc0-8781-4531-ad48-f9f8055a18ca None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:57.953014 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e042d-5ca1-4115-ba98-27c53edc3eb4 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:57.997927 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b5457a-d9b3-4cc8-9fd8-9d3310209de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:57.998706 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5457a-d9b3-4cc8-9fd8-9d3310209de1 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:57.999062 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5457a-d9b3-4cc8-9fd8-9d3310209de1 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:57.999343 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5457a-d9b3-4cc8-9fd8-9d3310209de1 None admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:58.000140 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b5457a-d9b3-4cc8-9fd8-9d3310209de1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:58.000450 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b5457a-d9b3-4cc8-9fd8-9d3310209de1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:58.001887 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b5457a-d9b3-4cc8-9fd8-9d3310209de1 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:58.021659 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 297/588] 10.222.0.98 () {56 vars in 1137 bytes} [Thu Aug 18 12:01:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:58.022324 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b5457a-d9b3-4cc8-9fd8-9d3310209de1 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:58.037456 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a5ddc0-8781-4531-ad48-f9f8055a18ca None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:58.051669 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:58.076689 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246b2d6-491b-40d5-b7ec-a1111c1807ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:58.077329 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246b2d6-491b-40d5-b7ec-a1111c1807ed None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:58.077613 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246b2d6-491b-40d5-b7ec-a1111c1807ed None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:58.077888 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246b2d6-491b-40d5-b7ec-a1111c1807ed None admin] PATH_INFO: `/v3/users/glance` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:58.088020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5246b2d6-491b-40d5-b7ec-a1111c1807ed None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:58.089525 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:58.090484 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5246b2d6-491b-40d5-b7ec-a1111c1807ed None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:58.102269 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-5246b2d6-491b-40d5-b7ec-a1111c1807ed None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 18 12:01:58.102269 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:58.102269 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:58.102269 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:58.102269 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:58.102269 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:58.102269 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:58.102269 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:58.102269 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:58.102269 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:58.102269 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:58.102269 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:58.102269 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 12:01:58.102269 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 12:01:58.102269 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 12:01:58.102269 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 12:01:58.109448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:58.109448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:58.109448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:01:58.109448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:01:58.109448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:01:58.109448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:01:58.109448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:58.109448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:58.109448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:58.109448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:58.109448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:58.109448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:58.109448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:58.109448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:58.109448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:58.109448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:58.109448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:58.109448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:58.110626 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:58.110626 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:58.110626 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 12:01:58.110626 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 12:01:58.110626 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 12:01:58.110626 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 12:01:58.110626 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 12:01:58.110626 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 12:01:58.110626 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 18 12:01:58.110626 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:58.110626 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 292/589] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:01:57 2022] GET /identity/v3/users/glance => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:58.110626 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246b2d6-491b-40d5-b7ec-a1111c1807ed None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:58.139089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b5457a-d9b3-4cc8-9fd8-9d3310209de1 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:58.177030 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b64ed-3f45-4cde-a393-682bd5621b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:58.178478 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b64ed-3f45-4cde-a393-682bd5621b81 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:58.178921 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b64ed-3f45-4cde-a393-682bd5621b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:58.179384 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b64ed-3f45-4cde-a393-682bd5621b81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:58.180291 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b64ed-3f45-4cde-a393-682bd5621b81 None admin] RBAC: Adding query filter params (role.id=563e3ed5c6164aae966b64cc302b114a, scope.project.id=6f676a5169464d83af4267db25025728, user.id=771591fe31cb4e8c9c4320c2a7b2719d) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:58.180965 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b64ed-3f45-4cde-a393-682bd5621b81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:58.183315 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b64ed-3f45-4cde-a393-682bd5621b81 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:58.191895 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 298/590] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:01:58 2022] GET /identity/v3/role_assignments?user.id=771591fe31cb4e8c9c4320c2a7b2719d&scope.project.id=6f676a5169464d83af4267db25025728&role.id=563e3ed5c6164aae966b64cc302b114a => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:58.193298 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b64ed-3f45-4cde-a393-682bd5621b81 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:58.224119 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246b2d6-491b-40d5-b7ec-a1111c1807ed None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:58.264047 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c9890-26e9-4b77-9d08-11735c8f53ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:58.264559 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c9890-26e9-4b77-9d08-11735c8f53ac None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:58.264858 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c9890-26e9-4b77-9d08-11735c8f53ac None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:58.265164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c9890-26e9-4b77-9d08-11735c8f53ac None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:58.265909 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250c9890-26e9-4b77-9d08-11735c8f53ac None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:58.266224 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250c9890-26e9-4b77-9d08-11735c8f53ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:58.267747 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250c9890-26e9-4b77-9d08-11735c8f53ac None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:58.292072 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 293/591] 10.222.0.98 () {56 vars in 1139 bytes} [Thu Aug 18 12:01:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:58.292072 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c9890-26e9-4b77-9d08-11735c8f53ac None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:58.325879 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:58.330084 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b64ed-3f45-4cde-a393-682bd5621b81 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:58.362926 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:58.372264 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b514f-62d0-4913-95d8-bc2c7e81f917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:58.373182 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b514f-62d0-4913-95d8-bc2c7e81f917 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:58.373471 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b514f-62d0-4913-95d8-bc2c7e81f917 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:58.373905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b514f-62d0-4913-95d8-bc2c7e81f917 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:58.386539 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663b514f-62d0-4913-95d8-bc2c7e81f917 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:58.388117 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663b514f-62d0-4913-95d8-bc2c7e81f917 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:58.404468 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-663b514f-62d0-4913-95d8-bc2c7e81f917 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:58.404468 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:58.404468 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:58.404468 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:58.404468 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:58.404468 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:58.404468 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:58.404468 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:58.404468 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:58.404468 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:58.404468 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:58.404468 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:58.404468 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:01:58.404468 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:01:58.404468 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:01:58.404468 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:01:58.405685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:58.405685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:58.405685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:58.405685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:58.405685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:58.405685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:58.405685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:58.405685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:58.405685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:58.405685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:58.405685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:58.405685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:58.405685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:58.405685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:58.405685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:58.405685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:58.405685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:01:58.407299 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:01:58.407299 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:58.407299 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:58.407299 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 299/592] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:01:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:58.407869 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46d2ca-0135-4243-8ecb-48f43450a450 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:58.407980 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46d2ca-0135-4243-8ecb-48f43450a450 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:58.408255 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46d2ca-0135-4243-8ecb-48f43450a450 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:58.413512 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 300/593] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:01:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:01:58.413512 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e46d2ca-0135-4243-8ecb-48f43450a450 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:58.416392 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c9890-26e9-4b77-9d08-11735c8f53ac None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:58.452909 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ce8eb-e7db-4849-a734-1ce0fa72c18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:58.454062 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ce8eb-e7db-4849-a734-1ce0fa72c18f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:58.454537 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ce8eb-e7db-4849-a734-1ce0fa72c18f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:58.454716 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ce8eb-e7db-4849-a734-1ce0fa72c18f None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:58.455438 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ce8eb-e7db-4849-a734-1ce0fa72c18f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:58.455772 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ce8eb-e7db-4849-a734-1ce0fa72c18f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:58.457218 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ce8eb-e7db-4849-a734-1ce0fa72c18f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:58.480522 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 294/594] 10.222.0.98 () {56 vars in 1137 bytes} [Thu Aug 18 12:01:58 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:58.480522 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ce8eb-e7db-4849-a734-1ce0fa72c18f None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:58.532466 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e46d2ca-0135-4243-8ecb-48f43450a450 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:58.565992 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c3e26a-010b-4be7-b331-b4e5fe951f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:58.566622 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3e26a-010b-4be7-b331-b4e5fe951f44 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:58.566904 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3e26a-010b-4be7-b331-b4e5fe951f44 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:58.567177 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3e26a-010b-4be7-b331-b4e5fe951f44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:58.579154 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c3e26a-010b-4be7-b331-b4e5fe951f44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:58.580577 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c3e26a-010b-4be7-b331-b4e5fe951f44 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:58.589046 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:58.596902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-52c3e26a-010b-4be7-b331-b4e5fe951f44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:58.596902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:58.596902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:58.596902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:58.596902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:58.596902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:58.596902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:58.596902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:58.596902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:58.596902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:58.596902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:58.596902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:58.596902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:01:58.596902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:01:58.596902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:01:58.596902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:01:58.603454 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:58.603454 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:58.603454 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:58.603454 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:58.603454 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:58.603454 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:58.603454 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:58.603454 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:58.603454 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:58.603454 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:58.603454 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:58.603454 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:58.603454 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:58.603454 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:58.603454 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:58.603454 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:58.603454 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:01:58.605987 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:01:58.605987 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:58.605987 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:58.605987 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 301/595] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:01:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:58.605987 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34901597-997a-4260-a558-2e0febabffa5 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:58.605987 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34901597-997a-4260-a558-2e0febabffa5 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:58.605987 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34901597-997a-4260-a558-2e0febabffa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:58.612510 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ce8eb-e7db-4849-a734-1ce0fa72c18f None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:58.620273 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-34901597-997a-4260-a558-2e0febabffa5 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:01:58.621499 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-34901597-997a-4260-a558-2e0febabffa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:01:58.652097 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a107be7-d04a-4acb-aced-0c0189468410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:58.652860 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a107be7-d04a-4acb-aced-0c0189468410 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:58.653805 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a107be7-d04a-4acb-aced-0c0189468410 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:58.653937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:58.654110 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a107be7-d04a-4acb-aced-0c0189468410 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:58.654216 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a107be7-d04a-4acb-aced-0c0189468410 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:58.654524 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a107be7-d04a-4acb-aced-0c0189468410 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:58.655972 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a107be7-d04a-4acb-aced-0c0189468410 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:58.668546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 295/596] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:01:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:58.669407 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a107be7-d04a-4acb-aced-0c0189468410 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:58.721136 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 302/597] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:01:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:01:58.721136 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34901597-997a-4260-a558-2e0febabffa5 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:58.790124 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a107be7-d04a-4acb-aced-0c0189468410 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:58.835017 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba150e-4c00-471a-9bab-2a2afd5049c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:58.835866 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba150e-4c00-471a-9bab-2a2afd5049c7 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:58.836498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba150e-4c00-471a-9bab-2a2afd5049c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:58.836874 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba150e-4c00-471a-9bab-2a2afd5049c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:58.852885 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ba150e-4c00-471a-9bab-2a2afd5049c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:58.853434 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34901597-997a-4260-a558-2e0febabffa5 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:58.854611 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ba150e-4c00-471a-9bab-2a2afd5049c7 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:58.866128 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:58.876199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-b0ba150e-4c00-471a-9bab-2a2afd5049c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:58.876199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:58.876199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:58.876199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:58.876199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:58.876199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:58.876199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:58.876199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:58.876199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:58.876199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:58.876199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:58.876199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:58.876199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:01:58.876199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:01:58.876199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:01:58.876199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:01:58.880226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:58.880226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:58.880226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:58.880226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:58.880226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:58.880226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:58.880226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:58.880226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:58.880226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:58.880226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:58.880226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:58.880226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:58.880226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:58.880226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:58.880226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:58.880226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:58.880226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:01:58.881102 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:01:58.881102 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:01:58.881102 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:58.881102 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 296/598] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:01:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:58.881102 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba150e-4c00-471a-9bab-2a2afd5049c7 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:58.900737 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e5c13b-0d0f-4045-ad76-e315741356ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:58.901555 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5c13b-0d0f-4045-ad76-e315741356ee None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:58.901944 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5c13b-0d0f-4045-ad76-e315741356ee None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:58.902438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5c13b-0d0f-4045-ad76-e315741356ee None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:58.903365 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e5c13b-0d0f-4045-ad76-e315741356ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:58.903949 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e5c13b-0d0f-4045-ad76-e315741356ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:58.905523 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e5c13b-0d0f-4045-ad76-e315741356ee None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:58.921305 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 303/599] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:01:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:58.922984 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e5c13b-0d0f-4045-ad76-e315741356ee None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:58.960354 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:59.005835 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba150e-4c00-471a-9bab-2a2afd5049c7 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:59.045443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fcbcb6-2a2b-400f-8bb7-c57f5c3f8a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:59.046103 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fcbcb6-2a2b-400f-8bb7-c57f5c3f8a3d None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:59.046350 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fcbcb6-2a2b-400f-8bb7-c57f5c3f8a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:59.046626 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fcbcb6-2a2b-400f-8bb7-c57f5c3f8a3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:59.052280 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e5c13b-0d0f-4045-ad76-e315741356ee None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:59.054505 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fcbcb6-2a2b-400f-8bb7-c57f5c3f8a3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:59.055998 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fcbcb6-2a2b-400f-8bb7-c57f5c3f8a3d None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:59.082510 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-c3fcbcb6-2a2b-400f-8bb7-c57f5c3f8a3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:01:59.082510 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:59.082510 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:59.082510 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:59.082510 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:59.082510 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:59.082510 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:59.082510 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:59.082510 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:59.082510 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:59.082510 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:59.082510 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:59.082510 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 12:01:59.082510 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 12:01:59.082510 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 12:01:59.083533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 12:01:59.083533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:59.083533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:59.083533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:59.083533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:59.083533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:59.083533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:59.083533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:59.083533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:59.083533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:59.083533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:59.083533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:59.083533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:59.083533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:59.083533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:59.083533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:59.083533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:59.084442 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 12:01:59.084442 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 12:01:59.084442 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:01:59.084442 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:01:59.084442 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:01:59.084442 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:01:59.084442 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:01:59.084442 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:59.084442 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 297/600] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:01:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:59.085294 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fcbcb6-2a2b-400f-8bb7-c57f5c3f8a3d None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:59.099648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f401a1c-4d9c-48a3-b2c7-a570c5e610b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:59.100251 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f401a1c-4d9c-48a3-b2c7-a570c5e610b7 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:59.100538 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f401a1c-4d9c-48a3-b2c7-a570c5e610b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:59.100824 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f401a1c-4d9c-48a3-b2c7-a570c5e610b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:59.101471 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f401a1c-4d9c-48a3-b2c7-a570c5e610b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:59.101789 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f401a1c-4d9c-48a3-b2c7-a570c5e610b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:59.103169 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f401a1c-4d9c-48a3-b2c7-a570c5e610b7 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:59.117847 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 304/601] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:01:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:59.118755 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f401a1c-4d9c-48a3-b2c7-a570c5e610b7 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:59.155379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:59.201693 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fcbcb6-2a2b-400f-8bb7-c57f5c3f8a3d None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:59.215849 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:59.235660 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f401a1c-4d9c-48a3-b2c7-a570c5e610b7 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:59.260870 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6eb88d-afd5-4023-af3a-42a72458f9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:59.261715 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6eb88d-afd5-4023-af3a-42a72458f9a3 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:59.261852 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6eb88d-afd5-4023-af3a-42a72458f9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:59.262067 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6eb88d-afd5-4023-af3a-42a72458f9a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:59.270084 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6eb88d-afd5-4023-af3a-42a72458f9a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:59.271592 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6eb88d-afd5-4023-af3a-42a72458f9a3 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:59.285049 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-db6eb88d-afd5-4023-af3a-42a72458f9a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:01:59.285049 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:59.285049 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:59.285049 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:59.285049 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:59.285049 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:59.285049 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:59.285049 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:59.285049 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:59.285049 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:59.285049 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:59.285049 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:59.285049 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 12:01:59.285049 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 12:01:59.285049 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 12:01:59.286214 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 12:01:59.286214 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:59.286214 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:59.286214 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:59.286214 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:59.286214 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:59.286214 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:59.286214 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:59.286214 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:59.286214 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:59.286214 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:59.286214 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:59.286214 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:59.286214 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:59.286214 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:59.286214 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:59.286214 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:59.292318 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 12:01:59.292318 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 12:01:59.292318 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:01:59.292318 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:01:59.292318 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:01:59.292318 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:01:59.292318 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:01:59.292318 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:01:59.292318 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 298/602] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:01:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:59.292318 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6eb88d-afd5-4023-af3a-42a72458f9a3 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:59.298823 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6471a17-6f42-44fa-b603-4f447b0dc19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:59.299602 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6471a17-6f42-44fa-b603-4f447b0dc19f None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:59.299969 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6471a17-6f42-44fa-b603-4f447b0dc19f None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:59.300375 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6471a17-6f42-44fa-b603-4f447b0dc19f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:59.314633 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6471a17-6f42-44fa-b603-4f447b0dc19f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:01:59.320301 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-e6471a17-6f42-44fa-b603-4f447b0dc19f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:01:59.320301 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:59.320301 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:59.320301 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:59.320301 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:59.320301 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:59.320301 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:59.320301 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:59.320301 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:59.320301 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:59.320301 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:59.320301 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:59.320301 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:01:59.320301 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:01:59.320301 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:01:59.320301 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:01:59.321448 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:01:59.321448 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:01:59.321448 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:01:59.321448 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:01:59.321448 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:59.321448 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:59.321448 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:59.321448 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:59.321448 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:59.321448 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:59.321448 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:59.321448 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:59.321448 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:59.321448 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:59.321448 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:59.321448 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:59.322573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:59.322573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:59.322573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:59.322573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:59.322573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:01:59.322573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:01:59.322573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:01:59.322573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:01:59.322573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:01:59.322573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:01:59.322573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:01:59.322573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:59.322573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 305/603] 10.222.0.98 () {56 vars in 1102 bytes} [Thu Aug 18 12:01:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:59.324281 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6471a17-6f42-44fa-b603-4f447b0dc19f None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:59.410664 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6eb88d-afd5-4023-af3a-42a72458f9a3 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:59.432969 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:59.434763 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6471a17-6f42-44fa-b603-4f447b0dc19f None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:59.449890 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310efc72-4b7b-4ecd-bb20-ba32b26901c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:59.450964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310efc72-4b7b-4ecd-bb20-ba32b26901c4 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:59.451295 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310efc72-4b7b-4ecd-bb20-ba32b26901c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:59.451556 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310efc72-4b7b-4ecd-bb20-ba32b26901c4 None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:59.452295 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310efc72-4b7b-4ecd-bb20-ba32b26901c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:59.452605 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310efc72-4b7b-4ecd-bb20-ba32b26901c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:59.454111 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310efc72-4b7b-4ecd-bb20-ba32b26901c4 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:59.467104 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 299/604] 10.222.0.98 () {56 vars in 1148 bytes} [Thu Aug 18 12:01:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:59.467996 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310efc72-4b7b-4ecd-bb20-ba32b26901c4 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:59.476217 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78970f8-361c-44d9-9a7b-86ecf7883c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:59.477290 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78970f8-361c-44d9-9a7b-86ecf7883c8e None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:59.477570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78970f8-361c-44d9-9a7b-86ecf7883c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:59.477850 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78970f8-361c-44d9-9a7b-86ecf7883c8e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:59.486183 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78970f8-361c-44d9-9a7b-86ecf7883c8e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:59.487665 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78970f8-361c-44d9-9a7b-86ecf7883c8e None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:59.499057 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-d78970f8-361c-44d9-9a7b-86ecf7883c8e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:01:59.499057 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:01:59.499057 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:01:59.499057 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:01:59.499057 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:01:59.499057 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:01:59.499057 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:01:59.499057 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:01:59.499057 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:01:59.499057 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:01:59.499057 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:01:59.499057 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:01:59.499057 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 12:01:59.499057 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 12:01:59.499057 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 12:01:59.500111 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 12:01:59.500111 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:01:59.500111 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:01:59.500111 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:01:59.500111 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:01:59.500111 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:01:59.500111 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:01:59.500111 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:01:59.500111 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:01:59.500111 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:01:59.500111 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:01:59.500111 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:01:59.500111 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:01:59.500111 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:01:59.500111 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:01:59.500111 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:01:59.500111 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:01:59.501542 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 12:01:59.501542 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 12:01:59.501542 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:01:59.501542 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:01:59.501542 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:01:59.501542 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:01:59.501542 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:01:59.501542 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:01:59.501542 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 306/605] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:01:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:01:59.501542 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78970f8-361c-44d9-9a7b-86ecf7883c8e None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:59.505571 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:59.584951 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310efc72-4b7b-4ecd-bb20-ba32b26901c4 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:59.620590 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78970f8-361c-44d9-9a7b-86ecf7883c8e None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:59.620791 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88482ece-14e8-41b5-88fd-3e7fc014410e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:59.621318 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88482ece-14e8-41b5-88fd-3e7fc014410e None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:59.621669 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88482ece-14e8-41b5-88fd-3e7fc014410e None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:59.621999 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88482ece-14e8-41b5-88fd-3e7fc014410e None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:59.622656 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88482ece-14e8-41b5-88fd-3e7fc014410e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:59.622990 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88482ece-14e8-41b5-88fd-3e7fc014410e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:59.624443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88482ece-14e8-41b5-88fd-3e7fc014410e None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:59.642274 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 300/606] 10.222.0.98 () {56 vars in 1148 bytes} [Thu Aug 18 12:01:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:59.643126 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88482ece-14e8-41b5-88fd-3e7fc014410e None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:59.666535 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1abc16-c115-476d-a3dd-bc0c3ae0c146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:59.667082 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1abc16-c115-476d-a3dd-bc0c3ae0c146 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:59.667343 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1abc16-c115-476d-a3dd-bc0c3ae0c146 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:59.667648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1abc16-c115-476d-a3dd-bc0c3ae0c146 None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:59.668315 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1abc16-c115-476d-a3dd-bc0c3ae0c146 None admin] RBAC: Adding query filter params (name=service) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:59.668624 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1abc16-c115-476d-a3dd-bc0c3ae0c146 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:59.670138 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1abc16-c115-476d-a3dd-bc0c3ae0c146 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:59.682309 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 307/607] 10.222.0.98 () {56 vars in 1103 bytes} [Thu Aug 18 12:01:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:59.683162 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1abc16-c115-476d-a3dd-bc0c3ae0c146 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:59.696628 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:59.758395 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:01:59.762735 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88482ece-14e8-41b5-88fd-3e7fc014410e None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:59.801049 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1abc16-c115-476d-a3dd-bc0c3ae0c146 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:01:59.803843 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea7f4a6-3ef0-4f32-b628-94b9dae9714c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:59.804523 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7f4a6-3ef0-4f32-b628-94b9dae9714c None admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:59.804811 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7f4a6-3ef0-4f32-b628-94b9dae9714c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:59.805126 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7f4a6-3ef0-4f32-b628-94b9dae9714c None admin] PATH_INFO: `/v3/projects/6f676a5169464d83af4267db25025728/users/c41b6fb3cb064a49b9236077a8faed54/roles/563e3ed5c6164aae966b64cc302b114a` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:59.820824 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea7f4a6-3ef0-4f32-b628-94b9dae9714c None admin] RBAC: Authorizing `identity:create_grant(project_id=6f676a5169464d83af4267db25025728, user_id=c41b6fb3cb064a49b9236077a8faed54, role_id=563e3ed5c6164aae966b64cc302b114a)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:59.822270 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea7f4a6-3ef0-4f32-b628-94b9dae9714c None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:59.838036 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a494657-1c72-42c9-9cb2-2a640a1bfe6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:01:59.839710 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a494657-1c72-42c9-9cb2-2a640a1bfe6e None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:01:59.839710 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a494657-1c72-42c9-9cb2-2a640a1bfe6e None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:01:59.839710 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a494657-1c72-42c9-9cb2-2a640a1bfe6e None admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:01:59.840028 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a494657-1c72-42c9-9cb2-2a640a1bfe6e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:01:59.840339 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a494657-1c72-42c9-9cb2-2a640a1bfe6e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:01:59.841770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a494657-1c72-42c9-9cb2-2a640a1bfe6e None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:01:59.855246 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 308/608] 10.222.0.98 () {56 vars in 1148 bytes} [Thu Aug 18 12:01:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:01:59.856149 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a494657-1c72-42c9-9cb2-2a640a1bfe6e None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:01:59.893368 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 301/609] 10.222.0.98 () {58 vars in 1439 bytes} [Thu Aug 18 12:01:59 2022] PUT /identity/v3/projects/6f676a5169464d83af4267db25025728/users/c41b6fb3cb064a49b9236077a8faed54/roles/563e3ed5c6164aae966b64cc302b114a => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:01:59.894528 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea7f4a6-3ef0-4f32-b628-94b9dae9714c None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:00.009166 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a494657-1c72-42c9-9cb2-2a640a1bfe6e None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:00.038347 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:00.042924 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:00.050784 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339fca57-1afc-4723-a645-2018b2684acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:00.052240 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339fca57-1afc-4723-a645-2018b2684acf None admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:00.052617 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339fca57-1afc-4723-a645-2018b2684acf None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:00.052981 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339fca57-1afc-4723-a645-2018b2684acf None admin] PATH_INFO: `/v3/projects/6f676a5169464d83af4267db25025728/users/185644affb284c2b91e604cf99803ffb/roles/563e3ed5c6164aae966b64cc302b114a` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:00.057810 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea7f4a6-3ef0-4f32-b628-94b9dae9714c None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:00.071397 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339fca57-1afc-4723-a645-2018b2684acf None admin] RBAC: Authorizing `identity:create_grant(project_id=6f676a5169464d83af4267db25025728, user_id=185644affb284c2b91e604cf99803ffb, role_id=563e3ed5c6164aae966b64cc302b114a)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:00.072910 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339fca57-1afc-4723-a645-2018b2684acf None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:00.103449 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836bb60-ed2a-4e03-bc14-d0206d9836c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:00.104314 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836bb60-ed2a-4e03-bc14-d0206d9836c9 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:00.104697 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836bb60-ed2a-4e03-bc14-d0206d9836c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:00.105072 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836bb60-ed2a-4e03-bc14-d0206d9836c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:00.116189 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c836bb60-ed2a-4e03-bc14-d0206d9836c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:00.117783 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c836bb60-ed2a-4e03-bc14-d0206d9836c9 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:00.134272 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-c836bb60-ed2a-4e03-bc14-d0206d9836c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:00.134272 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:00.134272 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:00.134272 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:00.134272 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:00.134272 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:00.134272 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:00.134272 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:00.134272 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:00.134272 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:00.134272 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:00.134272 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:00.134272 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:00.134272 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:00.134272 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:00.134272 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:00.135533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:00.135533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:00.135533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:00.135533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:00.135533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:00.135533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:00.135533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:00.135533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:00.135533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:00.135533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:00.135533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:00.135533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:00.135533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:00.135533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:00.135533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:00.135533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:00.135533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:00.136672 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:00.136672 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:00.136672 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:00.136672 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 302/610] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:01:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:00.136672 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836bb60-ed2a-4e03-bc14-d0206d9836c9 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:00.199450 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 309/611] 10.222.0.98 () {58 vars in 1439 bytes} [Thu Aug 18 12:01:59 2022] PUT /identity/v3/projects/6f676a5169464d83af4267db25025728/users/185644affb284c2b91e604cf99803ffb/roles/563e3ed5c6164aae966b64cc302b114a => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:02:00.199450 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c842f18-d5c0-4911-802d-be9a3d847bd3 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:00.199450 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c842f18-d5c0-4911-802d-be9a3d847bd3 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:00.199450 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c842f18-d5c0-4911-802d-be9a3d847bd3 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:00.199450 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 310/612] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:00.199450 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c842f18-d5c0-4911-802d-be9a3d847bd3 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:00.258597 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836bb60-ed2a-4e03-bc14-d0206d9836c9 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:00.298772 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5a020-6704-4d40-85c0-676e4a49e31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:00.299436 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5a020-6704-4d40-85c0-676e4a49e31e None admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:00.299712 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5a020-6704-4d40-85c0-676e4a49e31e None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:00.299983 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5a020-6704-4d40-85c0-676e4a49e31e None admin] PATH_INFO: `/v3/projects/6f676a5169464d83af4267db25025728/users/d020063702b0481e9f4111680766f7fc/roles/563e3ed5c6164aae966b64cc302b114a` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:00.317442 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:00.321046 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5a020-6704-4d40-85c0-676e4a49e31e None admin] RBAC: Authorizing `identity:create_grant(project_id=6f676a5169464d83af4267db25025728, user_id=d020063702b0481e9f4111680766f7fc, role_id=563e3ed5c6164aae966b64cc302b114a)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:00.322579 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5a020-6704-4d40-85c0-676e4a49e31e None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:00.348641 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c842f18-d5c0-4911-802d-be9a3d847bd3 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:00.385150 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1f06db-ef2f-46d9-a034-e68e8abde52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:00.385758 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f06db-ef2f-46d9-a034-e68e8abde52c None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:00.386038 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f06db-ef2f-46d9-a034-e68e8abde52c None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:00.386312 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f06db-ef2f-46d9-a034-e68e8abde52c None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:00.386950 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1f06db-ef2f-46d9-a034-e68e8abde52c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:00.387289 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1f06db-ef2f-46d9-a034-e68e8abde52c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:00.388735 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1f06db-ef2f-46d9-a034-e68e8abde52c None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:00.401786 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 311/613] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:00.402795 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2665f-a0c8-4a11-8d6e-d5a49d2a663e None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:00.403160 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2665f-a0c8-4a11-8d6e-d5a49d2a663e None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:00.403378 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2665f-a0c8-4a11-8d6e-d5a49d2a663e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:00.423203 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-33e2665f-a0c8-4a11-8d6e-d5a49d2a663e None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:00.424381 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-33e2665f-a0c8-4a11-8d6e-d5a49d2a663e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:00.431664 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 303/614] 10.222.0.98 () {58 vars in 1439 bytes} [Thu Aug 18 12:02:00 2022] PUT /identity/v3/projects/6f676a5169464d83af4267db25025728/users/d020063702b0481e9f4111680766f7fc/roles/563e3ed5c6164aae966b64cc302b114a => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:02:00.432785 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5a020-6704-4d40-85c0-676e4a49e31e None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:00.470837 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:00.538260 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 312/615] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:02:00.631629 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5a020-6704-4d40-85c0-676e4a49e31e None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:00.680381 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885df6a8-07ff-4174-aeea-e6efb9469474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:00.681039 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885df6a8-07ff-4174-aeea-e6efb9469474 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:00.681333 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885df6a8-07ff-4174-aeea-e6efb9469474 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:00.681616 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885df6a8-07ff-4174-aeea-e6efb9469474 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:00.691772 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885df6a8-07ff-4174-aeea-e6efb9469474 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:00.691772 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885df6a8-07ff-4174-aeea-e6efb9469474 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:00.704420 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-885df6a8-07ff-4174-aeea-e6efb9469474 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 12:02:00.704420 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:00.704420 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:00.704420 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:00.704420 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:00.704420 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:00.704420 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:00.704420 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:00.704420 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:00.704420 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:00.704420 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:00.704420 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:00.704420 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 12:02:00.704420 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 12:02:00.704420 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 12:02:00.704420 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 12:02:00.705592 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:00.705592 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:00.705592 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:02:00.705592 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:00.705592 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:02:00.705592 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:00.705592 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:00.705592 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:00.705592 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:00.705592 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:00.705592 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:00.705592 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:00.705592 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:00.705592 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:00.705592 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:00.705592 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:00.705592 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:00.705592 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:00.706737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:00.706737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:00.706737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 12:02:00.706737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 12:02:00.706737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 12:02:00.706737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 12:02:00.706737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 12:02:00.706737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 12:02:00.706737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 12:02:00.706737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:00.708165 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 304/616] 10.222.0.98 () {56 vars in 1093 bytes} [Thu Aug 18 12:02:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:00.713610 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2665f-a0c8-4a11-8d6e-d5a49d2a663e None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:00.821945 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2665f-a0c8-4a11-8d6e-d5a49d2a663e None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:00.856263 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ccc49-fce0-4aa5-a52b-e407d20e877c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:00.856970 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ccc49-fce0-4aa5-a52b-e407d20e877c None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:00.857325 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ccc49-fce0-4aa5-a52b-e407d20e877c None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:00.857673 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ccc49-fce0-4aa5-a52b-e407d20e877c None admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:00.858481 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ccc49-fce0-4aa5-a52b-e407d20e877c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:00.858863 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ccc49-fce0-4aa5-a52b-e407d20e877c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:00.860984 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ccc49-fce0-4aa5-a52b-e407d20e877c None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:00.879931 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 313/617] 10.222.0.98 () {56 vars in 1133 bytes} [Thu Aug 18 12:02:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:02:00.886065 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885df6a8-07ff-4174-aeea-e6efb9469474 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:00.962160 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ccc49-fce0-4aa5-a52b-e407d20e877c None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:00.964739 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:01.018979 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:01.021618 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885df6a8-07ff-4174-aeea-e6efb9469474 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:01.061545 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0d8a8-68a3-4b00-a075-0c09051ad573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:01.063132 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0d8a8-68a3-4b00-a075-0c09051ad573 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:01.063773 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0d8a8-68a3-4b00-a075-0c09051ad573 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:01.064200 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0d8a8-68a3-4b00-a075-0c09051ad573 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:01.152563 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a0d8a8-68a3-4b00-a075-0c09051ad573 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:01.153931 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a0d8a8-68a3-4b00-a075-0c09051ad573 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:01.191596 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ccc49-fce0-4aa5-a52b-e407d20e877c None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:01.251199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:01.263082 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6ab0f-46e9-4dad-bdd3-a931acfd512a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:01.263754 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6ab0f-46e9-4dad-bdd3-a931acfd512a None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:01.264042 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6ab0f-46e9-4dad-bdd3-a931acfd512a None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:01.264319 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6ab0f-46e9-4dad-bdd3-a931acfd512a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:01.280193 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd6ab0f-46e9-4dad-bdd3-a931acfd512a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:02:01.286091 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-4cd6ab0f-46e9-4dad-bdd3-a931acfd512a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:02:01.286091 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:01.286091 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:01.286091 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:01.286091 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:01.286091 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:01.286091 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:01.286091 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:01.286091 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:01.286091 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:01.286091 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:01.286091 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:01.286091 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:02:01.286091 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:02:01.286091 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:02:01.286091 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:02:01.287316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:02:01.287316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:02:01.287316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:02:01.287316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:02:01.287316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:01.287316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:01.287316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:01.287316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:01.287316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:01.287316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:01.287316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:01.287316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:01.287316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:01.287316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:01.287316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:01.287316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:01.288559 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:01.288559 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:01.288559 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:01.288559 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:01.288559 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:02:01.288559 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:02:01.288559 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:02:01.288559 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:02:01.288559 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:02:01.288559 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:02:01.288559 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:02:01.288559 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:01.288559 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 314/618] 10.222.0.98 () {56 vars in 1102 bytes} [Thu Aug 18 12:02:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:01.298312 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6ab0f-46e9-4dad-bdd3-a931acfd512a None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:01.346927 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 305/619] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:02:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:02:01.352810 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0d8a8-68a3-4b00-a075-0c09051ad573 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:01.480903 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:01.488850 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0d8a8-68a3-4b00-a075-0c09051ad573 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:01.533335 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6ab0f-46e9-4dad-bdd3-a931acfd512a None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:01.543276 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750ce29-934f-43b1-87d1-41f0fe297480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:01.543934 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750ce29-934f-43b1-87d1-41f0fe297480 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:01.544213 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750ce29-934f-43b1-87d1-41f0fe297480 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:01.544482 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750ce29-934f-43b1-87d1-41f0fe297480 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:01.561055 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6750ce29-934f-43b1-87d1-41f0fe297480 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:01.562488 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6750ce29-934f-43b1-87d1-41f0fe297480 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:01.586753 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-6750ce29-934f-43b1-87d1-41f0fe297480 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:01.586753 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:01.586753 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:01.586753 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:01.586753 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:01.586753 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:01.586753 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:01.586753 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:01.586753 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:01.586753 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:01.586753 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:01.586753 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:01.586753 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:01.586753 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:01.586753 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:01.586753 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:01.587977 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:01.587977 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:01.587977 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:01.587977 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:01.587977 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:01.587977 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:01.587977 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:01.587977 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:01.587977 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:01.587977 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:01.587977 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:01.587977 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:01.587977 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:01.587977 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:01.587977 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:01.587977 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:01.587977 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:01.589117 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:01.589117 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:01.589117 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:01.589117 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 306/620] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:02:01.595988 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750ce29-934f-43b1-87d1-41f0fe297480 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:01.596935 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c813e2-667c-4a67-86c3-0beee430c3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:01.597544 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c813e2-667c-4a67-86c3-0beee430c3dd None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:01.597819 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c813e2-667c-4a67-86c3-0beee430c3dd None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:01.598147 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c813e2-667c-4a67-86c3-0beee430c3dd None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:01.598814 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c813e2-667c-4a67-86c3-0beee430c3dd None admin] RBAC: Adding query filter params (name=service) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:01.599114 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c813e2-667c-4a67-86c3-0beee430c3dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:01.600502 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c813e2-667c-4a67-86c3-0beee430c3dd None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:01.619049 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 315/621] 10.222.0.98 () {56 vars in 1103 bytes} [Thu Aug 18 12:02:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:01.624215 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c813e2-667c-4a67-86c3-0beee430c3dd None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:01.631781 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:01.737082 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c813e2-667c-4a67-86c3-0beee430c3dd None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:01.783217 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438332e-d148-4956-af2e-5efcd106f498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:01.783911 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438332e-d148-4956-af2e-5efcd106f498 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:01.784192 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438332e-d148-4956-af2e-5efcd106f498 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:01.784461 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438332e-d148-4956-af2e-5efcd106f498 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:01.795837 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8438332e-d148-4956-af2e-5efcd106f498 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:01.797538 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8438332e-d148-4956-af2e-5efcd106f498 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:01.800296 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:01.816561 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-8438332e-d148-4956-af2e-5efcd106f498 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:01.816561 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:01.816561 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:01.816561 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:01.816561 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:01.816561 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:01.816561 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:01.816561 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:01.816561 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:01.816561 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:01.816561 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:01.816561 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:01.816561 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:01.816561 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:01.816561 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:01.816561 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:01.817792 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:01.817792 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:01.817792 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:01.817792 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:01.817792 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:01.817792 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:01.817792 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:01.817792 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:01.817792 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:01.817792 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:01.817792 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:01.817792 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:01.817792 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:01.817792 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:01.817792 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:01.817792 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:01.817792 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:01.818868 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:01.818868 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:01.818868 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:01.818868 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 316/622] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:02:01.824630 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438332e-d148-4956-af2e-5efcd106f498 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:01.840861 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750ce29-934f-43b1-87d1-41f0fe297480 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:01.926060 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba9fb5-5119-4e71-8e84-c36cc7f2bc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:01.928835 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba9fb5-5119-4e71-8e84-c36cc7f2bc93 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:01.929116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba9fb5-5119-4e71-8e84-c36cc7f2bc93 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:01.929389 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba9fb5-5119-4e71-8e84-c36cc7f2bc93 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:01.930042 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba9fb5-5119-4e71-8e84-c36cc7f2bc93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:01.930346 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba9fb5-5119-4e71-8e84-c36cc7f2bc93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:01.935527 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba9fb5-5119-4e71-8e84-c36cc7f2bc93 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:01.960162 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 307/623] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:01.966310 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba9fb5-5119-4e71-8e84-c36cc7f2bc93 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:02.061698 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438332e-d148-4956-af2e-5efcd106f498 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:02.089877 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:02.111013 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1853712-dab8-4e6a-8df0-39a55db3d71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:02.111902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1853712-dab8-4e6a-8df0-39a55db3d71b None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:02.112370 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1853712-dab8-4e6a-8df0-39a55db3d71b None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:02.112792 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1853712-dab8-4e6a-8df0-39a55db3d71b None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:02.113659 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1853712-dab8-4e6a-8df0-39a55db3d71b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:02.114168 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1853712-dab8-4e6a-8df0-39a55db3d71b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:02.115710 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1853712-dab8-4e6a-8df0-39a55db3d71b None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:02.130048 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 317/624] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:02.134638 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba9fb5-5119-4e71-8e84-c36cc7f2bc93 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:02.136433 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bfc0da-fc95-48db-ba96-22be76155d0c None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:02.136878 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bfc0da-fc95-48db-ba96-22be76155d0c None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:02.137343 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bfc0da-fc95-48db-ba96-22be76155d0c None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:02.139221 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 318/625] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:02.140106 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bfc0da-fc95-48db-ba96-22be76155d0c None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:02.179741 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e5ef6-2b69-46ea-b6d9-72100210ab87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:02.181197 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e5ef6-2b69-46ea-b6d9-72100210ab87 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:02.181634 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e5ef6-2b69-46ea-b6d9-72100210ab87 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:02.182080 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e5ef6-2b69-46ea-b6d9-72100210ab87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:02.191338 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030e5ef6-2b69-46ea-b6d9-72100210ab87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:02.194054 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030e5ef6-2b69-46ea-b6d9-72100210ab87 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:02.208064 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-030e5ef6-2b69-46ea-b6d9-72100210ab87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:02.208064 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:02.208064 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:02.208064 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:02.208064 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:02.208064 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:02.208064 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:02.208064 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:02.208064 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:02.208064 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:02.208064 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:02.208064 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:02.208064 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 12:02:02.208064 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 12:02:02.208064 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 12:02:02.209199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 12:02:02.209199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:02.209199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:02.209199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:02.209199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:02.209199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:02.209199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:02.209199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:02.209199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:02.209199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:02.209199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:02.209199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:02.209199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:02.209199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:02.209199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:02.209199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:02.209199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:02.210304 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 12:02:02.210304 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 12:02:02.210304 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:02:02.210304 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:02:02.210304 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:02:02.210304 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:02:02.210304 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:02.210304 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:02.212748 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 308/626] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:02:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:02.214175 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0efb70-c16e-4245-94a6-e091d877251e None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:02.214605 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0efb70-c16e-4245-94a6-e091d877251e None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:02.215488 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0efb70-c16e-4245-94a6-e091d877251e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:02.219562 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:02.239929 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-9d0efb70-c16e-4245-94a6-e091d877251e None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:02.241279 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-9d0efb70-c16e-4245-94a6-e091d877251e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:02.276031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bfc0da-fc95-48db-ba96-22be76155d0c None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:02.314581 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd1c08a-53e0-4472-b278-798eef600d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:02.315614 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1c08a-53e0-4472-b278-798eef600d1b None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:02.316047 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1c08a-53e0-4472-b278-798eef600d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:02.316470 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1c08a-53e0-4472-b278-798eef600d1b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:02.319494 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 309/627] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:02:02.320515 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0efb70-c16e-4245-94a6-e091d877251e None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:02.330946 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd1c08a-53e0-4472-b278-798eef600d1b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:02.332579 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd1c08a-53e0-4472-b278-798eef600d1b None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:02.345101 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-0cd1c08a-53e0-4472-b278-798eef600d1b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 18 12:02:02.345101 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:02.345101 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:02.345101 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:02.345101 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:02.345101 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:02.345101 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:02.345101 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:02.345101 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:02.345101 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:02.345101 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:02.345101 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:02.345101 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 12:02:02.345101 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 12:02:02.345101 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 12:02:02.345101 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 12:02:02.346347 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:02.346347 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:02.346347 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:02:02.346347 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:02.346347 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:02:02.346347 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:02.346347 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:02.346347 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:02.346347 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:02.346347 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:02.346347 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:02.346347 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:02.346347 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:02.346347 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:02.346347 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:02.346347 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:02.346347 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:02.346347 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:02.347544 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:02.347544 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:02.347544 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 12:02:02.347544 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 12:02:02.347544 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 12:02:02.347544 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 12:02:02.347544 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 12:02:02.347544 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 12:02:02.347544 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 18 12:02:02.347544 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:02.350922 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 319/628] 10.222.0.98 () {56 vars in 1096 bytes} [Thu Aug 18 12:02:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:02.352402 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0aa23a-9294-4877-b69c-6baeab1120a0 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:02.352698 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0aa23a-9294-4877-b69c-6baeab1120a0 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:02.352980 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0aa23a-9294-4877-b69c-6baeab1120a0 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:02.358463 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 320/629] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:02.359394 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0aa23a-9294-4877-b69c-6baeab1120a0 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:02.391907 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:02.457587 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0efb70-c16e-4245-94a6-e091d877251e None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:02.480480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0aa23a-9294-4877-b69c-6baeab1120a0 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:02.504209 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:02.504841 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3b9166-4e1e-45c2-90e0-b95edd1f4649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:02.505181 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b9166-4e1e-45c2-90e0-b95edd1f4649 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:02.505692 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b9166-4e1e-45c2-90e0-b95edd1f4649 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:02.506030 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b9166-4e1e-45c2-90e0-b95edd1f4649 None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:02.506740 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3b9166-4e1e-45c2-90e0-b95edd1f4649 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:02.507318 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3b9166-4e1e-45c2-90e0-b95edd1f4649 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:02.508873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3b9166-4e1e-45c2-90e0-b95edd1f4649 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:02.521717 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 310/630] 10.222.0.98 () {56 vars in 1148 bytes} [Thu Aug 18 12:02:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:02.527538 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b02dfb-f20c-4f92-b670-40636a3fb4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:02.528324 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee3ee4-71ff-4406-b398-13e6187a9443 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:02.528512 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b02dfb-f20c-4f92-b670-40636a3fb4ce None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:02.528869 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee3ee4-71ff-4406-b398-13e6187a9443 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:02.529080 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b02dfb-f20c-4f92-b670-40636a3fb4ce None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:02.529413 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee3ee4-71ff-4406-b398-13e6187a9443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:02.529670 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b02dfb-f20c-4f92-b670-40636a3fb4ce None admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:02.530540 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b02dfb-f20c-4f92-b670-40636a3fb4ce None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:02.530989 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b02dfb-f20c-4f92-b670-40636a3fb4ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:02.532981 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b02dfb-f20c-4f92-b670-40636a3fb4ce None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:02.549721 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-13ee3ee4-71ff-4406-b398-13e6187a9443 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:02.551109 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-13ee3ee4-71ff-4406-b398-13e6187a9443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:02.561037 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 321/631] 10.222.0.98 () {56 vars in 1135 bytes} [Thu Aug 18 12:02:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:02.561037 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b02dfb-f20c-4f92-b670-40636a3fb4ce None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:02.638938 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 311/632] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:02:02.640333 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee3ee4-71ff-4406-b398-13e6187a9443 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:02.697351 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b02dfb-f20c-4f92-b670-40636a3fb4ce None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:02.705952 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:02.746851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb9f987-c3ff-46d5-98bd-b0383966b052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:02.748131 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9f987-c3ff-46d5-98bd-b0383966b052 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:02.749051 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9f987-c3ff-46d5-98bd-b0383966b052 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:02.749535 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9f987-c3ff-46d5-98bd-b0383966b052 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:02.750677 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb9f987-c3ff-46d5-98bd-b0383966b052 None admin] RBAC: Adding query filter params (role.id=563e3ed5c6164aae966b64cc302b114a, scope.project.id=6f676a5169464d83af4267db25025728, user.id=171c72a6b29e418bbb4ea5c6faffe3b3) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:02.751958 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb9f987-c3ff-46d5-98bd-b0383966b052 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:02.753897 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb9f987-c3ff-46d5-98bd-b0383966b052 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:02.763458 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 322/633] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:02:02 2022] GET /identity/v3/role_assignments?user.id=171c72a6b29e418bbb4ea5c6faffe3b3&scope.project.id=6f676a5169464d83af4267db25025728&role.id=563e3ed5c6164aae966b64cc302b114a => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:02.764187 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee3ee4-71ff-4406-b398-13e6187a9443 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:02.764853 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb9f987-c3ff-46d5-98bd-b0383966b052 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:02.813401 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:02.822306 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc59bd-5cff-454b-9441-12f33f7b9c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:02.823654 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc59bd-5cff-454b-9441-12f33f7b9c11 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:02.824131 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc59bd-5cff-454b-9441-12f33f7b9c11 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:02.824676 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc59bd-5cff-454b-9441-12f33f7b9c11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:02.839974 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc59bd-5cff-454b-9441-12f33f7b9c11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:02.842018 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc59bd-5cff-454b-9441-12f33f7b9c11 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:02.866905 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-4ddc59bd-5cff-454b-9441-12f33f7b9c11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:02.866905 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:02.866905 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:02.866905 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:02.866905 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:02.866905 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:02.866905 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:02.866905 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:02.866905 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:02.866905 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:02.866905 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:02.866905 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:02.866905 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:02.866905 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:02.866905 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:02.866905 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:02.868134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:02.868134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:02.868134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:02.868134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:02.868134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:02.868134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:02.868134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:02.868134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:02.868134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:02.868134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:02.868134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:02.868134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:02.868134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:02.868134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:02.868134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:02.868134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:02.868134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:02.869240 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:02.869240 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:02.869240 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:02.876061 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 312/634] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:02.877358 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2be50-76c1-47b7-94bf-1796ce6e2c49 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:02.877358 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2be50-76c1-47b7-94bf-1796ce6e2c49 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:02.877558 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2be50-76c1-47b7-94bf-1796ce6e2c49 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:02.879083 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 313/635] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:02.879740 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae2be50-76c1-47b7-94bf-1796ce6e2c49 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:02.896455 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb9f987-c3ff-46d5-98bd-b0383966b052 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:02.945396 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6243a0-f097-4192-86f0-4a7317d52036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:02.946041 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6243a0-f097-4192-86f0-4a7317d52036 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:02.946325 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6243a0-f097-4192-86f0-4a7317d52036 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:02.946597 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6243a0-f097-4192-86f0-4a7317d52036 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:02.959974 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6243a0-f097-4192-86f0-4a7317d52036 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:02:02.966090 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-4b6243a0-f097-4192-86f0-4a7317d52036 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:02:02.966090 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:02.966090 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:02.966090 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:02.966090 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:02.966090 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:02.966090 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:02.966090 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:02.966090 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:02.966090 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:02.966090 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:02.966090 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:02.966090 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:02:02.966090 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:02:02.966090 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:02:02.966090 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:02:02.967016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:02:02.967016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:02:02.967016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:02:02.967016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:02:02.967016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:02.967016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:02.967016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:02.967016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:02.967016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:02.967016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:02.967016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:02.967016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:02.967016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:02.967016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:02.967016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:02.967016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:02.970464 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:02.970464 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:02.970464 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:02.970464 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:02.970464 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:02:02.970464 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:02:02.970464 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:02:02.970464 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:02:02.970464 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:02:02.970464 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:02:02.970464 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:02:02.970464 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:02.970464 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 323/636] 10.222.0.98 () {56 vars in 1102 bytes} [Thu Aug 18 12:02:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:02.970464 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ab3b1-e02b-4df6-864d-e7859a2a2502 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:02.971255 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ab3b1-e02b-4df6-864d-e7859a2a2502 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:02.971255 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ab3b1-e02b-4df6-864d-e7859a2a2502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:02.990170 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-b11ab3b1-e02b-4df6-864d-e7859a2a2502 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:02.991480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-b11ab3b1-e02b-4df6-864d-e7859a2a2502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:03.005613 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:03.012930 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae2be50-76c1-47b7-94bf-1796ce6e2c49 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:03.056984 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e640e-40a2-4b11-912d-10b8ef0dc58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:03.057740 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e640e-40a2-4b11-912d-10b8ef0dc58a None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:03.058082 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e640e-40a2-4b11-912d-10b8ef0dc58a None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:03.058471 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e640e-40a2-4b11-912d-10b8ef0dc58a None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:03.059123 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33e640e-40a2-4b11-912d-10b8ef0dc58a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:03.059507 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33e640e-40a2-4b11-912d-10b8ef0dc58a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:03.061128 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33e640e-40a2-4b11-912d-10b8ef0dc58a None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:03.069635 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 324/637] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:02:03.070934 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ab3b1-e02b-4df6-864d-e7859a2a2502 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:03.079598 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 314/638] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:03.081218 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e640e-40a2-4b11-912d-10b8ef0dc58a None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:03.107796 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:03.191331 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ab3b1-e02b-4df6-864d-e7859a2a2502 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:03.209845 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e640e-40a2-4b11-912d-10b8ef0dc58a None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:03.233561 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395858f7-a122-4eca-8890-063e476bbe3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:03.234297 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395858f7-a122-4eca-8890-063e476bbe3c None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:03.234652 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395858f7-a122-4eca-8890-063e476bbe3c None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:03.235079 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395858f7-a122-4eca-8890-063e476bbe3c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:03.252612 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395858f7-a122-4eca-8890-063e476bbe3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:02:03.257521 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21618ec3-ed6d-4549-8d77-1f894609207b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:03.257967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-395858f7-a122-4eca-8890-063e476bbe3c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:02:03.257967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:03.257967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:03.257967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:03.257967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:03.257967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:03.257967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:03.257967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:03.257967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:03.257967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:03.257967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:03.257967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:03.257967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:02:03.257967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:02:03.257967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:02:03.257967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:02:03.259065 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:02:03.259065 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:02:03.259065 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:02:03.259065 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:02:03.259065 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:03.259065 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:03.259065 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:03.259065 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:03.259065 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:03.259065 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:03.259065 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:03.259065 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:03.259065 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:03.259065 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:03.259065 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:03.259065 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:03.260186 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:03.260186 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:03.260186 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:03.260186 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:03.260186 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:02:03.260186 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:02:03.260186 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:02:03.260186 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:02:03.260186 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:02:03.260186 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:02:03.260186 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:02:03.260186 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:03.261446 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21618ec3-ed6d-4549-8d77-1f894609207b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:03.261867 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21618ec3-ed6d-4549-8d77-1f894609207b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:03.262260 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21618ec3-ed6d-4549-8d77-1f894609207b None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:03.263050 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21618ec3-ed6d-4549-8d77-1f894609207b None admin] RBAC: Adding query filter params (name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:03.263525 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21618ec3-ed6d-4549-8d77-1f894609207b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:03.265014 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21618ec3-ed6d-4549-8d77-1f894609207b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:03.266763 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 325/639] 10.222.0.98 () {56 vars in 1102 bytes} [Thu Aug 18 12:02:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:03.267731 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395858f7-a122-4eca-8890-063e476bbe3c None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:03.277199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 315/640] 10.222.0.98 () {56 vars in 1103 bytes} [Thu Aug 18 12:02:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:03.278241 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21618ec3-ed6d-4549-8d77-1f894609207b None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:03.305470 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:03.367847 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:03.379391 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395858f7-a122-4eca-8890-063e476bbe3c None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:03.398380 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21618ec3-ed6d-4549-8d77-1f894609207b None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:03.421889 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d811b12-5809-48a9-8ff0-45ee0d3c95ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:03.423340 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d811b12-5809-48a9-8ff0-45ee0d3c95ed None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:03.423667 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d811b12-5809-48a9-8ff0-45ee0d3c95ed None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:03.424339 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d811b12-5809-48a9-8ff0-45ee0d3c95ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:03.431272 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d811b12-5809-48a9-8ff0-45ee0d3c95ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:03.432751 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d811b12-5809-48a9-8ff0-45ee0d3c95ed None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:03.434882 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fcc3d0-01fc-4dfd-81f5-56d318b9e7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:03.435527 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fcc3d0-01fc-4dfd-81f5-56d318b9e7c8 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:03.435811 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fcc3d0-01fc-4dfd-81f5-56d318b9e7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:03.436087 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fcc3d0-01fc-4dfd-81f5-56d318b9e7c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:03.436756 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fcc3d0-01fc-4dfd-81f5-56d318b9e7c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:03.437066 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fcc3d0-01fc-4dfd-81f5-56d318b9e7c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:03.438510 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fcc3d0-01fc-4dfd-81f5-56d318b9e7c8 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:03.444089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-7d811b12-5809-48a9-8ff0-45ee0d3c95ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:03.444089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:03.444089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:03.444089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:03.444089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:03.444089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:03.444089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:03.444089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:03.444089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:03.444089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:03.444089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:03.444089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:03.444089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 12:02:03.444089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 12:02:03.444089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 12:02:03.445098 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 12:02:03.445098 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:03.445098 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:03.445098 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:03.445098 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:03.445098 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:03.445098 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:03.445098 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:03.445098 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:03.445098 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:03.445098 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:03.445098 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:03.445098 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:03.445098 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:03.445098 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:03.445098 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:03.445098 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:03.445994 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 12:02:03.445994 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 12:02:03.445994 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:02:03.445994 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:02:03.445994 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:02:03.445994 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:02:03.445994 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:03.445994 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:03.449897 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 326/641] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:02:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:03.450775 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d811b12-5809-48a9-8ff0-45ee0d3c95ed None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:03.450953 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 316/642] 10.222.0.98 () {56 vars in 1103 bytes} [Thu Aug 18 12:02:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:03.452360 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fcc3d0-01fc-4dfd-81f5-56d318b9e7c8 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:03.546544 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:03.564841 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d811b12-5809-48a9-8ff0-45ee0d3c95ed None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:03.567608 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fcc3d0-01fc-4dfd-81f5-56d318b9e7c8 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:03.601136 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3310de-4956-4f90-b523-fef535c3dfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:03.601767 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3310de-4956-4f90-b523-fef535c3dfca None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:03.602054 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3310de-4956-4f90-b523-fef535c3dfca None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:03.602337 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3310de-4956-4f90-b523-fef535c3dfca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:03.603490 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64698618-dfd5-4c19-8992-c0f1d110534f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:03.604183 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64698618-dfd5-4c19-8992-c0f1d110534f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:03.604469 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64698618-dfd5-4c19-8992-c0f1d110534f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:03.604748 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64698618-dfd5-4c19-8992-c0f1d110534f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:03.613582 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3310de-4956-4f90-b523-fef535c3dfca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:03.615034 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3310de-4956-4f90-b523-fef535c3dfca None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:03.616047 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64698618-dfd5-4c19-8992-c0f1d110534f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:02:03.622020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-64698618-dfd5-4c19-8992-c0f1d110534f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:02:03.622020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:03.622020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:03.622020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:03.622020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:03.622020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:03.622020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:03.622020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:03.622020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:03.622020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:03.622020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:03.622020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:03.622020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:02:03.622020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:02:03.622020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:02:03.622020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:02:03.623162 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:02:03.623162 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:02:03.623162 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:02:03.623162 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:02:03.623162 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:03.623162 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:03.623162 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:03.623162 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:03.623162 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:03.623162 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:03.623162 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:03.623162 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:03.623162 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:03.623162 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:03.623162 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:03.623162 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:03.624183 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:03.624183 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:03.624183 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:03.624183 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:03.624183 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:02:03.624183 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:02:03.624183 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:02:03.624183 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:02:03.624183 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:02:03.624183 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:02:03.624183 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:02:03.624183 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:03.624183 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 317/643] 10.222.0.98 () {56 vars in 1102 bytes} [Thu Aug 18 12:02:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:03.627393 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:03.628840 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64698618-dfd5-4c19-8992-c0f1d110534f None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:03.634089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-da3310de-4956-4f90-b523-fef535c3dfca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:03.634089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:03.634089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:03.634089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:03.634089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:03.634089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:03.634089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:03.634089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:03.634089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:03.634089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:03.634089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:03.634089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:03.634089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:03.634089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:03.634089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:03.634089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:03.640360 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:03.640360 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:03.640360 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:03.640360 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:03.640360 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:03.640360 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:03.640360 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:03.640360 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:03.640360 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:03.640360 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:03.640360 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:03.640360 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:03.640360 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:03.640360 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:03.640360 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:03.640360 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:03.640360 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:03.641464 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:03.641464 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:03.641464 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:03.641464 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 327/644] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:03.641464 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3310de-4956-4f90-b523-fef535c3dfca None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:03.746019 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64698618-dfd5-4c19-8992-c0f1d110534f None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:03.749464 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3310de-4956-4f90-b523-fef535c3dfca None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:03.783202 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba018ede-44cd-4fc2-a6c5-d08e4b4ec52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:03.784077 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba018ede-44cd-4fc2-a6c5-d08e4b4ec52f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:03.784535 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba018ede-44cd-4fc2-a6c5-d08e4b4ec52f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:03.784958 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba018ede-44cd-4fc2-a6c5-d08e4b4ec52f None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:03.785796 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba018ede-44cd-4fc2-a6c5-d08e4b4ec52f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:03.786252 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba018ede-44cd-4fc2-a6c5-d08e4b4ec52f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:03.786481 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7674c093-e081-4161-9a77-234340a48c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:03.787115 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674c093-e081-4161-9a77-234340a48c28 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:03.787400 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674c093-e081-4161-9a77-234340a48c28 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:03.787709 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674c093-e081-4161-9a77-234340a48c28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:03.788505 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba018ede-44cd-4fc2-a6c5-d08e4b4ec52f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:03.798662 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7674c093-e081-4161-9a77-234340a48c28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:03.799740 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:03.800166 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7674c093-e081-4161-9a77-234340a48c28 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:03.804349 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 318/645] 10.222.0.98 () {56 vars in 1148 bytes} [Thu Aug 18 12:02:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:03.805349 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba018ede-44cd-4fc2-a6c5-d08e4b4ec52f None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:03.815003 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-7674c093-e081-4161-9a77-234340a48c28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:03.815003 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:03.815003 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:03.815003 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:03.815003 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:03.815003 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:03.815003 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:03.815003 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:03.815003 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:03.815003 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:03.815003 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:03.815003 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:03.815003 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:03.815003 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:03.815003 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:03.815003 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:03.818311 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:03.818311 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:03.818311 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:03.818311 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:03.818311 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:03.818311 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:03.818311 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:03.818311 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:03.818311 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:03.818311 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:03.818311 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:03.818311 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:03.818311 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:03.818311 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:03.818311 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:03.818311 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:03.818311 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:03.819211 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:03.819211 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:03.819211 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:03.819211 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 328/646] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:03.819211 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7674c093-e081-4161-9a77-234340a48c28 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:03.886249 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:03.930413 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba018ede-44cd-4fc2-a6c5-d08e4b4ec52f None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:03.939452 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7674c093-e081-4161-9a77-234340a48c28 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:03.967876 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784f3a40-39c5-4bf9-b39d-5d54971bec49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:03.968498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f3a40-39c5-4bf9-b39d-5d54971bec49 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:03.968787 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f3a40-39c5-4bf9-b39d-5d54971bec49 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:03.969200 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f3a40-39c5-4bf9-b39d-5d54971bec49 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:03.970383 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784f3a40-39c5-4bf9-b39d-5d54971bec49 None admin] RBAC: Adding query filter params (name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:03.970694 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784f3a40-39c5-4bf9-b39d-5d54971bec49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:03.972093 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784f3a40-39c5-4bf9-b39d-5d54971bec49 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:03.976905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50117a6b-ae15-49a2-8b57-404cd9e1e0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:03.977521 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50117a6b-ae15-49a2-8b57-404cd9e1e0ef None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:03.977806 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50117a6b-ae15-49a2-8b57-404cd9e1e0ef None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:03.978084 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50117a6b-ae15-49a2-8b57-404cd9e1e0ef None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:03.978722 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50117a6b-ae15-49a2-8b57-404cd9e1e0ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:03.979027 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50117a6b-ae15-49a2-8b57-404cd9e1e0ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:03.980428 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50117a6b-ae15-49a2-8b57-404cd9e1e0ef None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:03.981922 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 319/647] 10.222.0.98 () {56 vars in 1103 bytes} [Thu Aug 18 12:02:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:03.982481 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784f3a40-39c5-4bf9-b39d-5d54971bec49 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:04.002589 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 329/648] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:04.002589 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50117a6b-ae15-49a2-8b57-404cd9e1e0ef None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:04.058356 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:04.100852 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784f3a40-39c5-4bf9-b39d-5d54971bec49 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:04.128113 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50117a6b-ae15-49a2-8b57-404cd9e1e0ef None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:04.136532 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567d5b32-6aba-48aa-be12-1e44b500ffc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:04.137189 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d5b32-6aba-48aa-be12-1e44b500ffc9 None admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:04.137474 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d5b32-6aba-48aa-be12-1e44b500ffc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:04.137747 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d5b32-6aba-48aa-be12-1e44b500ffc9 None admin] PATH_INFO: `/v3/projects/6f676a5169464d83af4267db25025728/users/771591fe31cb4e8c9c4320c2a7b2719d/roles/563e3ed5c6164aae966b64cc302b114a` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:04.146508 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:04.159719 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567d5b32-6aba-48aa-be12-1e44b500ffc9 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f676a5169464d83af4267db25025728, user_id=771591fe31cb4e8c9c4320c2a7b2719d, role_id=563e3ed5c6164aae966b64cc302b114a)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:04.161206 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567d5b32-6aba-48aa-be12-1e44b500ffc9 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:04.169210 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfd58de-3896-4815-801d-e604443978e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:04.169822 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd58de-3896-4815-801d-e604443978e1 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:04.170101 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd58de-3896-4815-801d-e604443978e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:04.170373 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd58de-3896-4815-801d-e604443978e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:04.171011 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfd58de-3896-4815-801d-e604443978e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:04.171317 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfd58de-3896-4815-801d-e604443978e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:04.173280 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfd58de-3896-4815-801d-e604443978e1 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:04.184702 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 330/649] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:04.196944 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfd58de-3896-4815-801d-e604443978e1 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:04.287377 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfd58de-3896-4815-801d-e604443978e1 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:04.320549 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5acad50-f630-4280-a215-6e343c41a40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:04.321183 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5acad50-f630-4280-a215-6e343c41a40f None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:04.321468 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5acad50-f630-4280-a215-6e343c41a40f None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:04.321788 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5acad50-f630-4280-a215-6e343c41a40f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:04.332137 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5acad50-f630-4280-a215-6e343c41a40f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:04.333688 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5acad50-f630-4280-a215-6e343c41a40f None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:04.347908 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-b5acad50-f630-4280-a215-6e343c41a40f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:04.347908 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:04.347908 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:04.347908 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:04.347908 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:04.347908 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:04.347908 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:04.347908 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:04.347908 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:04.347908 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:04.347908 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:04.347908 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:04.347908 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:04.347908 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:04.347908 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:04.347908 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:04.348920 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:04.348920 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:04.348920 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:04.348920 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:04.348920 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:04.348920 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:04.348920 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:04.348920 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:04.348920 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:04.348920 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:04.348920 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:04.348920 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:04.348920 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:04.348920 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:04.348920 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:04.348920 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:04.348920 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:04.349895 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:04.349895 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:04.349895 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:04.349895 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 331/650] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:04.349895 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5acad50-f630-4280-a215-6e343c41a40f None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:04.452879 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5acad50-f630-4280-a215-6e343c41a40f None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:04.486177 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89772d62-3a5b-4318-a914-4b6f138bee1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:04.486809 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89772d62-3a5b-4318-a914-4b6f138bee1b None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:04.487092 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89772d62-3a5b-4318-a914-4b6f138bee1b None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:04.487367 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89772d62-3a5b-4318-a914-4b6f138bee1b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:04.494777 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89772d62-3a5b-4318-a914-4b6f138bee1b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:04.496266 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89772d62-3a5b-4318-a914-4b6f138bee1b None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:04.507305 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-89772d62-3a5b-4318-a914-4b6f138bee1b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 12:02:04.507305 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:04.507305 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:04.507305 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:04.507305 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:04.507305 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:04.507305 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:04.507305 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:04.507305 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:04.507305 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:04.507305 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:04.507305 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:04.507305 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 12:02:04.507305 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 12:02:04.507305 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 12:02:04.507305 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 12:02:04.508656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:04.508656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:04.508656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:02:04.508656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:04.508656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:02:04.508656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:04.508656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:04.508656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:04.508656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:04.508656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:04.508656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:04.508656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:04.508656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:04.508656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:04.508656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:04.508656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:04.508656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:04.508656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:04.509631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:04.509631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:04.509631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 12:02:04.509631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 12:02:04.509631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 12:02:04.509631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 12:02:04.509631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 12:02:04.509631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 12:02:04.509631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 12:02:04.509631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:04.509631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 332/651] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:02:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:04.509631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89772d62-3a5b-4318-a914-4b6f138bee1b None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:04.540133 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:04.579444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 320/652] 10.222.0.98 () {58 vars in 1439 bytes} [Thu Aug 18 12:02:03 2022] PUT /identity/v3/projects/6f676a5169464d83af4267db25025728/users/771591fe31cb4e8c9c4320c2a7b2719d/roles/563e3ed5c6164aae966b64cc302b114a => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:02:04.579444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567d5b32-6aba-48aa-be12-1e44b500ffc9 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:04.620393 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:04.639509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89772d62-3a5b-4318-a914-4b6f138bee1b None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:04.693024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77355b1-05b4-4910-8871-3f0e64c6cd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:04.693884 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77355b1-05b4-4910-8871-3f0e64c6cd86 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:04.694341 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77355b1-05b4-4910-8871-3f0e64c6cd86 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:04.694716 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77355b1-05b4-4910-8871-3f0e64c6cd86 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:04.707887 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77355b1-05b4-4910-8871-3f0e64c6cd86 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:04.710091 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77355b1-05b4-4910-8871-3f0e64c6cd86 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:04.726387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-b77355b1-05b4-4910-8871-3f0e64c6cd86 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 18 12:02:04.726387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:04.726387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:04.726387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:04.726387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:04.726387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:04.726387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:04.726387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:04.726387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:04.726387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:04.726387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:04.726387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:04.726387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 12:02:04.726387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 12:02:04.726387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 12:02:04.726387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 12:02:04.727507 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:04.727507 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:04.727507 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:02:04.727507 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:04.727507 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:02:04.727507 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:04.727507 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:04.727507 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:04.727507 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:04.727507 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:04.727507 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:04.727507 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:04.727507 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:04.727507 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:04.727507 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:04.727507 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:04.727507 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:04.727507 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:04.728448 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:04.728448 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:04.728448 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 12:02:04.728448 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 12:02:04.728448 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 12:02:04.728448 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 12:02:04.728448 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 12:02:04.728448 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 12:02:04.728448 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 18 12:02:04.728448 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:04.728448 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 333/653] 10.222.0.98 () {56 vars in 1102 bytes} [Thu Aug 18 12:02:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:04.729070 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77355b1-05b4-4910-8871-3f0e64c6cd86 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:04.729715 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567d5b32-6aba-48aa-be12-1e44b500ffc9 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:04.747553 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:04.776869 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9a69f-01b0-4409-9843-d40d66240e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:04.777558 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9a69f-01b0-4409-9843-d40d66240e16 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:04.777842 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9a69f-01b0-4409-9843-d40d66240e16 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:04.778119 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9a69f-01b0-4409-9843-d40d66240e16 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:04.778834 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b9a69f-01b0-4409-9843-d40d66240e16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:04.779147 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b9a69f-01b0-4409-9843-d40d66240e16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:04.780833 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b9a69f-01b0-4409-9843-d40d66240e16 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:04.800143 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 321/654] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:04.800143 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9a69f-01b0-4409-9843-d40d66240e16 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:04.854606 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77355b1-05b4-4910-8871-3f0e64c6cd86 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:04.893475 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f69d1a-1d3c-4b96-a58d-d39904daee62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:04.894365 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f69d1a-1d3c-4b96-a58d-d39904daee62 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:04.894653 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f69d1a-1d3c-4b96-a58d-d39904daee62 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:04.894929 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f69d1a-1d3c-4b96-a58d-d39904daee62 None admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:04.895862 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f69d1a-1d3c-4b96-a58d-d39904daee62 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:04.896448 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f69d1a-1d3c-4b96-a58d-d39904daee62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:04.897906 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f69d1a-1d3c-4b96-a58d-d39904daee62 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:04.912072 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9a69f-01b0-4409-9843-d40d66240e16 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:04.917693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 334/655] 10.222.0.98 () {56 vars in 1137 bytes} [Thu Aug 18 12:02:04 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:04.918841 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f69d1a-1d3c-4b96-a58d-d39904daee62 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:04.925345 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:04.949079 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6998f0c3-0673-42e1-80af-b83905b034b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:04.949700 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998f0c3-0673-42e1-80af-b83905b034b9 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:04.950340 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998f0c3-0673-42e1-80af-b83905b034b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:04.950340 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998f0c3-0673-42e1-80af-b83905b034b9 None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:04.951125 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6998f0c3-0673-42e1-80af-b83905b034b9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:04.951471 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6998f0c3-0673-42e1-80af-b83905b034b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:04.953029 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6998f0c3-0673-42e1-80af-b83905b034b9 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:04.972399 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 322/656] 10.222.0.98 () {56 vars in 1139 bytes} [Thu Aug 18 12:02:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:04.973733 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60556130-424f-4f3d-a52d-3cf69a14d6be None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:04.974021 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60556130-424f-4f3d-a52d-3cf69a14d6be None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:04.974293 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60556130-424f-4f3d-a52d-3cf69a14d6be None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:04.979737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 323/657] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:04.979737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60556130-424f-4f3d-a52d-3cf69a14d6be None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:05.015793 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:05.044549 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f69d1a-1d3c-4b96-a58d-d39904daee62 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:05.086131 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc691ce-892f-4e67-a9e9-11a7bc683189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:05.086771 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc691ce-892f-4e67-a9e9-11a7bc683189 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:05.087055 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc691ce-892f-4e67-a9e9-11a7bc683189 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:05.087458 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc691ce-892f-4e67-a9e9-11a7bc683189 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:05.094922 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc691ce-892f-4e67-a9e9-11a7bc683189 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:05.095886 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60556130-424f-4f3d-a52d-3cf69a14d6be None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:05.096396 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc691ce-892f-4e67-a9e9-11a7bc683189 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:05.110304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-adc691ce-892f-4e67-a9e9-11a7bc683189 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 18 12:02:05.110304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:05.110304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:05.110304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:05.110304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:05.110304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:05.110304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:05.110304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:05.110304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:05.110304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:05.110304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:05.110304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:05.110304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 12:02:05.110304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 12:02:05.110304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 12:02:05.110304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 12:02:05.111312 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:05.111312 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:05.111312 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:02:05.111312 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:05.111312 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:02:05.111312 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:05.111312 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:05.111312 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:05.111312 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:05.111312 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:05.111312 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:05.111312 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:05.111312 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:05.111312 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:05.111312 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:05.111312 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:05.111312 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:05.111312 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:05.112341 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:05.112341 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:05.112341 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 12:02:05.112341 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 12:02:05.112341 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 12:02:05.112341 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 12:02:05.112341 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 12:02:05.112341 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 12:02:05.112341 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 18 12:02:05.112341 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:05.112341 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 335/658] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:02:04 2022] GET /identity/v3/users/glance => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:05.113456 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e06a9e-dc68-490e-b68e-8c3d8b8f22b6 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:05.113456 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e06a9e-dc68-490e-b68e-8c3d8b8f22b6 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:05.113456 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e06a9e-dc68-490e-b68e-8c3d8b8f22b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:05.132107 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-e7e06a9e-dc68-490e-b68e-8c3d8b8f22b6 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:05.133323 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-e7e06a9e-dc68-490e-b68e-8c3d8b8f22b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:05.137074 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965545fd-b89c-436c-b141-6115572a7f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:05.137893 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965545fd-b89c-436c-b141-6115572a7f57 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:05.138377 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965545fd-b89c-436c-b141-6115572a7f57 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:05.138841 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965545fd-b89c-436c-b141-6115572a7f57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:05.191844 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:05.202340 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 336/659] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:02:05.203238 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e06a9e-dc68-490e-b68e-8c3d8b8f22b6 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:05.231600 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965545fd-b89c-436c-b141-6115572a7f57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:05.234401 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965545fd-b89c-436c-b141-6115572a7f57 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:05.275280 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:05.319109 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e06a9e-dc68-490e-b68e-8c3d8b8f22b6 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:05.358292 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316a94d-178c-4450-86d5-f58bd2196fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:05.359188 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316a94d-178c-4450-86d5-f58bd2196fec None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:05.359695 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316a94d-178c-4450-86d5-f58bd2196fec None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:05.360738 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316a94d-178c-4450-86d5-f58bd2196fec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:05.363397 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 324/660] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:02:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:02:05.364374 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965545fd-b89c-436c-b141-6115572a7f57 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:05.437704 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:05.450624 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e316a94d-178c-4450-86d5-f58bd2196fec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:05.452943 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e316a94d-178c-4450-86d5-f58bd2196fec None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:05.469844 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965545fd-b89c-436c-b141-6115572a7f57 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:05.502924 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:05.510981 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f46782-2766-4098-a5e6-dd70d32d83fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:05.511644 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f46782-2766-4098-a5e6-dd70d32d83fc None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:05.511930 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f46782-2766-4098-a5e6-dd70d32d83fc None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:05.512216 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f46782-2766-4098-a5e6-dd70d32d83fc None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:05.513029 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f46782-2766-4098-a5e6-dd70d32d83fc None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:05.513266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f46782-2766-4098-a5e6-dd70d32d83fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:05.514693 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f46782-2766-4098-a5e6-dd70d32d83fc None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:05.533787 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 325/661] 10.222.0.98 () {56 vars in 1137 bytes} [Thu Aug 18 12:02:05 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:05.534665 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f46782-2766-4098-a5e6-dd70d32d83fc None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:05.586550 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 337/662] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:02:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:02:05.587408 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316a94d-178c-4450-86d5-f58bd2196fec None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:05.653480 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f46782-2766-4098-a5e6-dd70d32d83fc None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:05.692892 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd960a6c-1051-4bf4-af1e-3e848c442b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:05.694396 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd960a6c-1051-4bf4-af1e-3e848c442b9f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:05.694499 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd960a6c-1051-4bf4-af1e-3e848c442b9f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:05.694887 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd960a6c-1051-4bf4-af1e-3e848c442b9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:05.697131 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:05.706731 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316a94d-178c-4450-86d5-f58bd2196fec None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:05.710340 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd960a6c-1051-4bf4-af1e-3e848c442b9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:05.711924 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd960a6c-1051-4bf4-af1e-3e848c442b9f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:05.733670 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-fd960a6c-1051-4bf4-af1e-3e848c442b9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:05.733670 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:05.733670 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:05.733670 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:05.733670 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:05.733670 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:05.733670 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:05.733670 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:05.733670 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:05.733670 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:05.733670 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:05.733670 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:05.733670 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:05.733670 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:05.733670 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:05.733670 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:05.734704 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:05.734704 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:05.734704 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:05.734704 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:05.734704 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:05.734704 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:05.734704 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:05.734704 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:05.734704 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:05.734704 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:05.734704 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:05.734704 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:05.734704 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:05.734704 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:05.734704 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:05.734704 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:05.734704 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:05.736348 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:05.736348 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:05.736348 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:05.736348 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 326/663] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:05.736348 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd960a6c-1051-4bf4-af1e-3e848c442b9f None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:05.747778 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a19acd-3859-4ea7-84f4-7e3189debc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:05.748423 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a19acd-3859-4ea7-84f4-7e3189debc78 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:05.748707 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a19acd-3859-4ea7-84f4-7e3189debc78 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:05.748984 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a19acd-3859-4ea7-84f4-7e3189debc78 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:05.759777 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a19acd-3859-4ea7-84f4-7e3189debc78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:02:05.764399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-f7a19acd-3859-4ea7-84f4-7e3189debc78 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:02:05.764399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:05.764399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:05.764399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:05.764399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:05.764399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:05.764399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:05.764399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:05.764399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:05.764399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:05.764399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:05.764399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:05.764399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:02:05.764399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:02:05.764399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:02:05.764399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:02:05.765353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:02:05.765353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:02:05.765353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:02:05.765353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:02:05.765353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:05.765353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:05.765353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:05.765353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:05.765353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:05.765353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:05.765353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:05.765353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:05.765353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:05.765353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:05.765353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:05.765353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:05.766779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:05.766779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:05.766779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:05.766779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:05.766779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:02:05.766779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:02:05.766779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:02:05.766779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:02:05.766779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:02:05.766779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:02:05.766779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:02:05.766779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:05.766779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 338/664] 10.222.0.98 () {56 vars in 1096 bytes} [Thu Aug 18 12:02:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:05.767545 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a19acd-3859-4ea7-84f4-7e3189debc78 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:05.799665 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:05.871629 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd960a6c-1051-4bf4-af1e-3e848c442b9f None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:05.872927 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a19acd-3859-4ea7-84f4-7e3189debc78 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:05.909387 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348a52b-eb89-4529-8bc1-6eda50d1cc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:05.910024 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348a52b-eb89-4529-8bc1-6eda50d1cc27 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:05.910396 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348a52b-eb89-4529-8bc1-6eda50d1cc27 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:05.910733 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348a52b-eb89-4529-8bc1-6eda50d1cc27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:05.912383 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5a0d2-f485-49e0-abef-0e1f342150e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:05.912916 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5a0d2-f485-49e0-abef-0e1f342150e8 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:05.913396 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5a0d2-f485-49e0-abef-0e1f342150e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:05.913750 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5a0d2-f485-49e0-abef-0e1f342150e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:05.926969 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5a0d2-f485-49e0-abef-0e1f342150e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:05.928472 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5a0d2-f485-49e0-abef-0e1f342150e8 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:05.946273 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-16a5a0d2-f485-49e0-abef-0e1f342150e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:05.946273 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:05.946273 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:05.946273 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:05.946273 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:05.946273 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:05.946273 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:05.946273 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:05.946273 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:05.946273 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:05.946273 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:05.946273 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:05.946273 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:05.946273 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:05.946273 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:05.946273 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:05.947394 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:05.947394 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:05.947394 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:05.947394 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:05.947394 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:05.947394 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:05.947394 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:05.947394 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:05.947394 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:05.947394 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:05.947394 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:05.947394 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:05.947394 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:05.947394 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:05.947394 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:05.947394 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:05.947394 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:05.948326 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:05.948326 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:05.948326 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:05.948326 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 339/665] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:05.949093 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5a0d2-f485-49e0-abef-0e1f342150e8 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:05.952917 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:05.998862 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348a52b-eb89-4529-8bc1-6eda50d1cc27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:06.001435 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348a52b-eb89-4529-8bc1-6eda50d1cc27 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:06.047282 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:06.067328 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5a0d2-f485-49e0-abef-0e1f342150e8 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:06.115215 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54b0c9-4e66-4674-a550-966773fa2cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:06.115870 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54b0c9-4e66-4674-a550-966773fa2cc9 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:06.116162 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54b0c9-4e66-4674-a550-966773fa2cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:06.116442 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54b0c9-4e66-4674-a550-966773fa2cc9 None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:06.117097 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54b0c9-4e66-4674-a550-966773fa2cc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:06.117409 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54b0c9-4e66-4674-a550-966773fa2cc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:06.118792 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54b0c9-4e66-4674-a550-966773fa2cc9 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:06.127643 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 327/666] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:02:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:02:06.129248 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348a52b-eb89-4529-8bc1-6eda50d1cc27 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:06.132620 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 340/667] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:06.133375 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54b0c9-4e66-4674-a550-966773fa2cc9 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:06.211655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:06.260320 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348a52b-eb89-4529-8bc1-6eda50d1cc27 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:06.262008 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54b0c9-4e66-4674-a550-966773fa2cc9 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:06.302031 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd111a-1f2c-4412-be03-efa28d5bf250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:06.302982 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd111a-1f2c-4412-be03-efa28d5bf250 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:06.303182 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5869354-c25a-4240-9876-1c6c1d5276a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:06.303685 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd111a-1f2c-4412-be03-efa28d5bf250 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:06.303841 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5869354-c25a-4240-9876-1c6c1d5276a0 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:06.304127 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5869354-c25a-4240-9876-1c6c1d5276a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:06.304406 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5869354-c25a-4240-9876-1c6c1d5276a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:06.304606 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd111a-1f2c-4412-be03-efa28d5bf250 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:06.305055 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5869354-c25a-4240-9876-1c6c1d5276a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:06.305363 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5869354-c25a-4240-9876-1c6c1d5276a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:06.305798 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cd111a-1f2c-4412-be03-efa28d5bf250 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:06.306243 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cd111a-1f2c-4412-be03-efa28d5bf250 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:06.306742 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5869354-c25a-4240-9876-1c6c1d5276a0 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:06.308002 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cd111a-1f2c-4412-be03-efa28d5bf250 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:06.323491 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 341/668] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:06.323491 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5869354-c25a-4240-9876-1c6c1d5276a0 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:06.323946 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 328/669] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:02:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:06.325110 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd111a-1f2c-4412-be03-efa28d5bf250 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:06.328118 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:06.443586 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5869354-c25a-4240-9876-1c6c1d5276a0 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:06.452341 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd111a-1f2c-4412-be03-efa28d5bf250 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:06.470116 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:06.482748 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991224d8-f709-454a-903b-ea85581bae43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:06.483661 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991224d8-f709-454a-903b-ea85581bae43 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:06.484118 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991224d8-f709-454a-903b-ea85581bae43 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:06.485804 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991224d8-f709-454a-903b-ea85581bae43 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:06.495975 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991224d8-f709-454a-903b-ea85581bae43 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:06.498075 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991224d8-f709-454a-903b-ea85581bae43 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:06.501653 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f9abd3-bd0c-419b-96ff-03599fe3f3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:06.502294 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9abd3-bd0c-419b-96ff-03599fe3f3ba None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:06.502578 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9abd3-bd0c-419b-96ff-03599fe3f3ba None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:06.505924 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9abd3-bd0c-419b-96ff-03599fe3f3ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:06.513604 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-991224d8-f709-454a-903b-ea85581bae43 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:06.513604 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:06.513604 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:06.513604 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:06.513604 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:06.513604 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:06.513604 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:06.513604 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:06.513604 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:06.513604 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:06.513604 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:06.513604 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:06.513604 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 12:02:06.513604 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 12:02:06.513604 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 12:02:06.515765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 12:02:06.515765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:06.515765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:06.515765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:06.515765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:06.515765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:06.515765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:06.515765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:06.515765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:06.515765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:06.515765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:06.515765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:06.515765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:06.515765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:06.515765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:06.515765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:06.515765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:06.518136 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 12:02:06.518136 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 12:02:06.518136 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:02:06.518136 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:02:06.518136 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:02:06.518136 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:02:06.518136 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:06.518136 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:06.518136 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 342/670] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:02:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:06.518136 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991224d8-f709-454a-903b-ea85581bae43 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:06.525232 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f9abd3-bd0c-419b-96ff-03599fe3f3ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:06.526787 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f9abd3-bd0c-419b-96ff-03599fe3f3ba None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:06.548797 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-e4f9abd3-bd0c-419b-96ff-03599fe3f3ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:06.548797 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:06.548797 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:06.548797 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:06.548797 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:06.548797 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:06.548797 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:06.548797 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:06.548797 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:06.548797 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:06.548797 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:06.548797 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:06.548797 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:06.548797 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:06.548797 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:06.548797 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:06.551520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:06.551520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:06.551520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:06.551520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:06.551520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:06.551520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:06.551520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:06.551520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:06.551520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:06.551520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:06.551520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:06.551520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:06.551520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:06.551520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:06.551520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:06.551520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:06.551520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:06.554693 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:06.554693 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:06.554693 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:06.555081 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 329/671] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:06.556001 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f9abd3-bd0c-419b-96ff-03599fe3f3ba None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:06.628337 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:06.635352 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991224d8-f709-454a-903b-ea85581bae43 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:06.669594 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cbb5f4-d368-401b-b73a-990c33bbcf90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:06.670220 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cbb5f4-d368-401b-b73a-990c33bbcf90 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:06.670509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cbb5f4-d368-401b-b73a-990c33bbcf90 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:06.670771 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cbb5f4-d368-401b-b73a-990c33bbcf90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:06.675947 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f9abd3-bd0c-419b-96ff-03599fe3f3ba None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:06.683956 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cbb5f4-d368-401b-b73a-990c33bbcf90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:06.685418 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cbb5f4-d368-401b-b73a-990c33bbcf90 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:06.700152 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-c0cbb5f4-d368-401b-b73a-990c33bbcf90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:06.700152 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:06.700152 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:06.700152 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:06.700152 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:06.700152 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:06.700152 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:06.700152 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:06.700152 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:06.700152 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:06.700152 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:06.700152 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:06.700152 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:06.700152 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:06.700152 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:06.700152 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:06.701607 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:06.701607 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:06.701607 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:06.701607 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:06.701607 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:06.701607 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:06.701607 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:06.701607 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:06.701607 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:06.701607 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:06.701607 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:06.701607 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:06.701607 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:06.701607 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:06.701607 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:06.701607 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:06.701607 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:06.702718 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:06.702718 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:06.702718 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:06.702718 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 343/672] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:06.702718 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cbb5f4-d368-401b-b73a-990c33bbcf90 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:06.718238 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d1620-80f7-430b-bb1b-8f3ffc447168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:06.718863 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d1620-80f7-430b-bb1b-8f3ffc447168 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:06.719139 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d1620-80f7-430b-bb1b-8f3ffc447168 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:06.719492 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d1620-80f7-430b-bb1b-8f3ffc447168 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:06.726646 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469d1620-80f7-430b-bb1b-8f3ffc447168 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:06.728058 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469d1620-80f7-430b-bb1b-8f3ffc447168 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:06.738223 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:06.739273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-469d1620-80f7-430b-bb1b-8f3ffc447168 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:06.739273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:06.739273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:06.739273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:06.739273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:06.739273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:06.739273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:06.739273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:06.739273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:06.739273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:06.739273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:06.739273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:06.739273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 12:02:06.739273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 12:02:06.739273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 12:02:06.745251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 12:02:06.745251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:06.745251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:06.745251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:06.745251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:06.745251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:06.745251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:06.745251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:06.745251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:06.745251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:06.745251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:06.745251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:06.745251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:06.745251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:06.745251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:06.745251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:06.745251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:06.750343 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 12:02:06.750343 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 12:02:06.750343 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:02:06.750343 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:02:06.750343 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:02:06.750343 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:02:06.750343 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:06.750343 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:06.750343 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 330/673] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:02:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:06.750343 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d1620-80f7-430b-bb1b-8f3ffc447168 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:06.804475 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cbb5f4-d368-401b-b73a-990c33bbcf90 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:06.840225 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e74c66-3b91-4485-a90c-7760d7b537a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:06.840835 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e74c66-3b91-4485-a90c-7760d7b537a6 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:06.841114 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e74c66-3b91-4485-a90c-7760d7b537a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:06.841382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e74c66-3b91-4485-a90c-7760d7b537a6 None admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:06.841485 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d1620-80f7-430b-bb1b-8f3ffc447168 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:06.842068 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e74c66-3b91-4485-a90c-7760d7b537a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:06.842370 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e74c66-3b91-4485-a90c-7760d7b537a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:06.843826 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e74c66-3b91-4485-a90c-7760d7b537a6 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:06.855692 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:06.863554 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 344/674] 10.222.0.98 () {56 vars in 1148 bytes} [Thu Aug 18 12:02:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:06.863554 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73355a-e66b-4d73-a57b-a6f75ceeb5e4 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:06.863554 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73355a-e66b-4d73-a57b-a6f75ceeb5e4 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:06.863554 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73355a-e66b-4d73-a57b-a6f75ceeb5e4 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:06.863554 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 345/675] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:06.863554 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff73355a-e66b-4d73-a57b-a6f75ceeb5e4 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:06.877139 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ecab3-12f8-41bb-a1d0-b4199a426581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:06.877749 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ecab3-12f8-41bb-a1d0-b4199a426581 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:06.878032 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ecab3-12f8-41bb-a1d0-b4199a426581 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:06.878303 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ecab3-12f8-41bb-a1d0-b4199a426581 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:06.878999 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ecab3-12f8-41bb-a1d0-b4199a426581 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:06.879309 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ecab3-12f8-41bb-a1d0-b4199a426581 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:06.880691 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ecab3-12f8-41bb-a1d0-b4199a426581 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:06.898676 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 331/676] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:06.898676 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ecab3-12f8-41bb-a1d0-b4199a426581 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:06.963127 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:06.970444 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff73355a-e66b-4d73-a57b-a6f75ceeb5e4 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:06.997805 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ecab3-12f8-41bb-a1d0-b4199a426581 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:07.004622 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b6e3fe-ea15-49ef-8307-4457868aee02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:07.005233 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6e3fe-ea15-49ef-8307-4457868aee02 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:07.005517 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6e3fe-ea15-49ef-8307-4457868aee02 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:07.005797 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6e3fe-ea15-49ef-8307-4457868aee02 None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:07.006439 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b6e3fe-ea15-49ef-8307-4457868aee02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:07.006746 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b6e3fe-ea15-49ef-8307-4457868aee02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:07.008164 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b6e3fe-ea15-49ef-8307-4457868aee02 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:07.020316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 346/677] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:07.021157 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b6e3fe-ea15-49ef-8307-4457868aee02 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:07.036766 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f434e9e-b821-42b7-9fe9-9d46f1737d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:07.037372 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f434e9e-b821-42b7-9fe9-9d46f1737d27 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:07.037657 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f434e9e-b821-42b7-9fe9-9d46f1737d27 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:07.037954 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f434e9e-b821-42b7-9fe9-9d46f1737d27 None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:07.038631 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f434e9e-b821-42b7-9fe9-9d46f1737d27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:07.038971 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f434e9e-b821-42b7-9fe9-9d46f1737d27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:07.040418 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f434e9e-b821-42b7-9fe9-9d46f1737d27 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:07.052383 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 332/678] 10.222.0.98 () {56 vars in 1148 bytes} [Thu Aug 18 12:02:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:07.053584 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e799f-c5d4-4c43-8e2e-59b111fed95f None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:07.053880 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e799f-c5d4-4c43-8e2e-59b111fed95f None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:07.054162 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e799f-c5d4-4c43-8e2e-59b111fed95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:07.072773 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-ba2e799f-c5d4-4c43-8e2e-59b111fed95f None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:07.073979 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-ba2e799f-c5d4-4c43-8e2e-59b111fed95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:07.100687 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:07.122553 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b6e3fe-ea15-49ef-8307-4457868aee02 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:07.146808 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 333/679] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:02:07.147856 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e799f-c5d4-4c43-8e2e-59b111fed95f None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:07.157887 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbdca43-67cd-4196-9a1f-542da352883d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:07.158698 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbdca43-67cd-4196-9a1f-542da352883d None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:07.159136 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbdca43-67cd-4196-9a1f-542da352883d None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:07.159441 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbdca43-67cd-4196-9a1f-542da352883d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:07.160193 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbdca43-67cd-4196-9a1f-542da352883d None admin] RBAC: Adding query filter params (role.id=563e3ed5c6164aae966b64cc302b114a, scope.project.id=6f676a5169464d83af4267db25025728, user.id=c41b6fb3cb064a49b9236077a8faed54) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:07.160570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbdca43-67cd-4196-9a1f-542da352883d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:07.161943 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbdca43-67cd-4196-9a1f-542da352883d None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:07.169866 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 347/680] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:02:07 2022] GET /identity/v3/role_assignments?user.id=c41b6fb3cb064a49b9236077a8faed54&scope.project.id=6f676a5169464d83af4267db25025728&role.id=563e3ed5c6164aae966b64cc302b114a => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:07.171189 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbdca43-67cd-4196-9a1f-542da352883d None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:07.216333 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:07.293022 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e799f-c5d4-4c43-8e2e-59b111fed95f None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:07.318849 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbdca43-67cd-4196-9a1f-542da352883d None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:07.336637 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f051f5-9108-409b-bc89-a7d42370fdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:07.337632 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f051f5-9108-409b-bc89-a7d42370fdd3 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:07.338211 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f051f5-9108-409b-bc89-a7d42370fdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:07.338535 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f051f5-9108-409b-bc89-a7d42370fdd3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:07.346571 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f051f5-9108-409b-bc89-a7d42370fdd3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:07.348913 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f051f5-9108-409b-bc89-a7d42370fdd3 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:07.362506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-22f051f5-9108-409b-bc89-a7d42370fdd3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:07.362506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:07.362506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:07.362506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:07.362506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:07.362506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:07.362506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:07.362506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:07.362506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:07.362506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:07.362506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:07.362506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:07.362506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 12:02:07.362506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 12:02:07.362506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 12:02:07.363473 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 12:02:07.363473 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:07.363473 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:07.363473 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:07.363473 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:07.363473 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:07.363473 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:07.363473 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:07.363473 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:07.363473 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:07.363473 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:07.363473 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:07.363473 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:07.363473 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:07.363473 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:07.363473 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:07.363473 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:07.364379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 12:02:07.364379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 12:02:07.364379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:02:07.364379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:02:07.364379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:02:07.364379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:02:07.364379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:07.364379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:07.364379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 334/681] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:02:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:07.364865 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f051f5-9108-409b-bc89-a7d42370fdd3 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:07.365741 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bcdae7-3ede-4e4a-be73-f89d126565f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:07.366784 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bcdae7-3ede-4e4a-be73-f89d126565f5 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:07.368746 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bcdae7-3ede-4e4a-be73-f89d126565f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:07.368746 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bcdae7-3ede-4e4a-be73-f89d126565f5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:07.383848 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bcdae7-3ede-4e4a-be73-f89d126565f5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:07.386043 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bcdae7-3ede-4e4a-be73-f89d126565f5 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:07.399934 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-f3bcdae7-3ede-4e4a-be73-f89d126565f5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 12:02:07.399934 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:07.399934 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:07.399934 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:07.399934 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:07.399934 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:07.399934 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:07.399934 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:07.399934 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:07.399934 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:07.399934 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:07.399934 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:07.399934 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 12:02:07.399934 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 12:02:07.399934 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 12:02:07.399934 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 12:02:07.400901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:07.400901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:07.400901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:02:07.400901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:07.400901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:02:07.400901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:07.400901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:07.400901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:07.400901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:07.400901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:07.400901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:07.400901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:07.400901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:07.400901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:07.400901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:07.400901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:07.400901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:07.400901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:07.401846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:07.401846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:07.401846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 12:02:07.401846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 12:02:07.401846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 12:02:07.401846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 12:02:07.401846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 12:02:07.401846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 12:02:07.401846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 12:02:07.401846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:07.402385 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:07.402484 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 348/682] 10.222.0.98 () {56 vars in 1093 bytes} [Thu Aug 18 12:02:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:07.402484 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bcdae7-3ede-4e4a-be73-f89d126565f5 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:07.493101 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:07.496384 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f051f5-9108-409b-bc89-a7d42370fdd3 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:07.512944 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bcdae7-3ede-4e4a-be73-f89d126565f5 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:07.539695 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2265c-b03a-4e5c-8a12-c90a84d69102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:07.540307 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2265c-b03a-4e5c-8a12-c90a84d69102 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:07.540588 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2265c-b03a-4e5c-8a12-c90a84d69102 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:07.540917 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2265c-b03a-4e5c-8a12-c90a84d69102 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:07.541641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee2265c-b03a-4e5c-8a12-c90a84d69102 None admin] RBAC: Adding query filter params (role.id=563e3ed5c6164aae966b64cc302b114a, scope.project.id=6f676a5169464d83af4267db25025728, user.id=185644affb284c2b91e604cf99803ffb) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:07.541939 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee2265c-b03a-4e5c-8a12-c90a84d69102 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:07.543430 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee2265c-b03a-4e5c-8a12-c90a84d69102 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:07.551252 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a040e2f3-dbd5-454f-8852-36968ae60143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:07.552285 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a040e2f3-dbd5-454f-8852-36968ae60143 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:07.552772 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a040e2f3-dbd5-454f-8852-36968ae60143 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:07.553250 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a040e2f3-dbd5-454f-8852-36968ae60143 None admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:07.554473 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a040e2f3-dbd5-454f-8852-36968ae60143 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:07.555201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a040e2f3-dbd5-454f-8852-36968ae60143 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:07.557259 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a040e2f3-dbd5-454f-8852-36968ae60143 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:07.559617 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 335/683] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:02:07 2022] GET /identity/v3/role_assignments?user.id=185644affb284c2b91e604cf99803ffb&scope.project.id=6f676a5169464d83af4267db25025728&role.id=563e3ed5c6164aae966b64cc302b114a => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:07.560613 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2265c-b03a-4e5c-8a12-c90a84d69102 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:07.573530 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 349/684] 10.222.0.98 () {56 vars in 1148 bytes} [Thu Aug 18 12:02:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:07.574647 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a040e2f3-dbd5-454f-8852-36968ae60143 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:07.697399 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:07.707027 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a040e2f3-dbd5-454f-8852-36968ae60143 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:07.713777 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2265c-b03a-4e5c-8a12-c90a84d69102 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:07.745827 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890942f-606c-43cb-9d03-7e05b4149b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:07.746467 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890942f-606c-43cb-9d03-7e05b4149b31 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:07.746752 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890942f-606c-43cb-9d03-7e05b4149b31 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:07.747052 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890942f-606c-43cb-9d03-7e05b4149b31 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:07.756063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:07.757515 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f6347-80c1-40d2-bd70-77613827c10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:07.758210 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f6347-80c1-40d2-bd70-77613827c10b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:07.758554 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f6347-80c1-40d2-bd70-77613827c10b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:07.758910 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f6347-80c1-40d2-bd70-77613827c10b None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:07.759212 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7890942f-606c-43cb-9d03-7e05b4149b31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:02:07.759820 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f6347-80c1-40d2-bd70-77613827c10b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:07.760193 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f6347-80c1-40d2-bd70-77613827c10b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:07.761687 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f6347-80c1-40d2-bd70-77613827c10b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:07.764570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-7890942f-606c-43cb-9d03-7e05b4149b31 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:02:07.764570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:07.764570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:07.764570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:07.764570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:07.764570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:07.764570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:07.764570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:07.764570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:07.764570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:07.764570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:07.764570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:07.764570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:02:07.764570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:02:07.764570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:02:07.764570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:02:07.771382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:02:07.771382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:02:07.771382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:02:07.771382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:02:07.771382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:07.771382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:07.771382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:07.771382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:07.771382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:07.771382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:07.771382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:07.771382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:07.771382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:07.771382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:07.771382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:07.771382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:07.772568 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:07.772568 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:07.772568 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:07.772568 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:07.772568 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:02:07.772568 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:02:07.772568 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:02:07.772568 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:02:07.772568 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:02:07.772568 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:02:07.772568 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:02:07.772568 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:07.772568 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 350/685] 10.222.0.98 () {56 vars in 1102 bytes} [Thu Aug 18 12:02:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:07.772568 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890942f-606c-43cb-9d03-7e05b4149b31 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:07.787239 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 336/686] 10.222.0.98 () {56 vars in 1133 bytes} [Thu Aug 18 12:02:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:07.788380 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f6347-80c1-40d2-bd70-77613827c10b None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:07.881637 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890942f-606c-43cb-9d03-7e05b4149b31 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:07.916561 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f6347-80c1-40d2-bd70-77613827c10b None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:07.922593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc18986-d739-4ae1-bb1a-cc7cb21add05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:07.929738 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc18986-d739-4ae1-bb1a-cc7cb21add05 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:07.929738 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc18986-d739-4ae1-bb1a-cc7cb21add05 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:07.929738 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc18986-d739-4ae1-bb1a-cc7cb21add05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:07.929738 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc18986-d739-4ae1-bb1a-cc7cb21add05 None admin] RBAC: Adding query filter params (role.id=563e3ed5c6164aae966b64cc302b114a, scope.project.id=6f676a5169464d83af4267db25025728, user.id=d020063702b0481e9f4111680766f7fc) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:07.929738 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc18986-d739-4ae1-bb1a-cc7cb21add05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:07.929738 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc18986-d739-4ae1-bb1a-cc7cb21add05 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:07.936695 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 351/687] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:02:07 2022] GET /identity/v3/role_assignments?user.id=d020063702b0481e9f4111680766f7fc&scope.project.id=6f676a5169464d83af4267db25025728&role.id=563e3ed5c6164aae966b64cc302b114a => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:07.938239 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc18986-d739-4ae1-bb1a-cc7cb21add05 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:07.960244 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d371b53-c4c8-42b7-8106-ceabedce4931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:07.961356 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d371b53-c4c8-42b7-8106-ceabedce4931 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:07.961713 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d371b53-c4c8-42b7-8106-ceabedce4931 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:07.962272 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d371b53-c4c8-42b7-8106-ceabedce4931 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:07.962956 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d371b53-c4c8-42b7-8106-ceabedce4931 None admin] RBAC: Adding query filter params (name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:07.963261 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d371b53-c4c8-42b7-8106-ceabedce4931 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:07.965137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d371b53-c4c8-42b7-8106-ceabedce4931 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:07.977947 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 337/688] 10.222.0.98 () {56 vars in 1103 bytes} [Thu Aug 18 12:02:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:07.983975 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d371b53-c4c8-42b7-8106-ceabedce4931 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:08.005619 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:08.059064 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:08.142948 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc18986-d739-4ae1-bb1a-cc7cb21add05 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:08.168452 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d371b53-c4c8-42b7-8106-ceabedce4931 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:08.183120 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89f7e2f-bab9-4458-8c7b-49cab21dd27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:08.183786 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f7e2f-bab9-4458-8c7b-49cab21dd27d None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:08.184073 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f7e2f-bab9-4458-8c7b-49cab21dd27d None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:08.184351 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f7e2f-bab9-4458-8c7b-49cab21dd27d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:08.209014 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0950e8-3a3b-47bd-8da4-3a708389fdc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:08.209807 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0950e8-3a3b-47bd-8da4-3a708389fdc2 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:08.210092 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0950e8-3a3b-47bd-8da4-3a708389fdc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:08.210367 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0950e8-3a3b-47bd-8da4-3a708389fdc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:08.223550 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0950e8-3a3b-47bd-8da4-3a708389fdc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:08.225170 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0950e8-3a3b-47bd-8da4-3a708389fdc2 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:08.242717 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-6b0950e8-3a3b-47bd-8da4-3a708389fdc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:08.242717 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:08.242717 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:08.242717 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:08.242717 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:08.242717 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:08.242717 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:08.242717 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:08.242717 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:08.242717 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:08.242717 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:08.242717 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:08.242717 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:08.242717 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:08.242717 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:08.242717 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:08.244996 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:08.244996 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:08.244996 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:08.244996 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:08.244996 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:08.244996 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:08.244996 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:08.244996 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:08.244996 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:08.244996 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:08.244996 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:08.244996 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:08.244996 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:08.244996 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:08.244996 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:08.244996 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:08.244996 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:08.246960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:08.246960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:08.246960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:08.246960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 338/689] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:08.252417 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0950e8-3a3b-47bd-8da4-3a708389fdc2 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:08.266065 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89f7e2f-bab9-4458-8c7b-49cab21dd27d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:08.267589 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89f7e2f-bab9-4458-8c7b-49cab21dd27d None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:08.330385 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:08.355803 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:08.384008 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0950e8-3a3b-47bd-8da4-3a708389fdc2 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:08.389627 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 352/690] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:02:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:02:08.391666 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89f7e2f-bab9-4458-8c7b-49cab21dd27d None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:08.432056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728d91b-54bb-492b-9797-2ab77b3c8b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:08.433133 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728d91b-54bb-492b-9797-2ab77b3c8b3f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:08.433733 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728d91b-54bb-492b-9797-2ab77b3c8b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:08.434124 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728d91b-54bb-492b-9797-2ab77b3c8b3f None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:08.434900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a728d91b-54bb-492b-9797-2ab77b3c8b3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:08.435523 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a728d91b-54bb-492b-9797-2ab77b3c8b3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:08.437729 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a728d91b-54bb-492b-9797-2ab77b3c8b3f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:08.455531 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 339/691] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:08.460960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728d91b-54bb-492b-9797-2ab77b3c8b3f None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:08.536944 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89f7e2f-bab9-4458-8c7b-49cab21dd27d None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:08.591450 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728d91b-54bb-492b-9797-2ab77b3c8b3f None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:08.604179 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2acaa-46a9-4bd2-a630-7c0d6deafc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:08.605173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2acaa-46a9-4bd2-a630-7c0d6deafc07 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:08.605739 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2acaa-46a9-4bd2-a630-7c0d6deafc07 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:08.606199 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2acaa-46a9-4bd2-a630-7c0d6deafc07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:08.619800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:08.628431 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc2acaa-46a9-4bd2-a630-7c0d6deafc07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:08.630036 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc2acaa-46a9-4bd2-a630-7c0d6deafc07 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:08.636391 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d83a16-24a5-4408-be75-88af24b663e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:08.638931 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d83a16-24a5-4408-be75-88af24b663e0 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:08.638931 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d83a16-24a5-4408-be75-88af24b663e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:08.638931 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d83a16-24a5-4408-be75-88af24b663e0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:08.653615 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d83a16-24a5-4408-be75-88af24b663e0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:08.655085 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d83a16-24a5-4408-be75-88af24b663e0 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:08.662060 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-2bc2acaa-46a9-4bd2-a630-7c0d6deafc07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:08.662060 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:08.662060 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:08.662060 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:08.662060 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:08.662060 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:08.662060 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:08.662060 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:08.662060 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:08.662060 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:08.662060 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:08.662060 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:08.662060 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:08.662060 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:08.662060 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:08.662060 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:08.663177 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:08.663177 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:08.663177 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:08.663177 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:08.663177 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:08.663177 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:08.663177 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:08.663177 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:08.663177 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:08.663177 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:08.663177 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:08.663177 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:08.663177 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:08.663177 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:08.663177 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:08.663177 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:08.663177 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:08.664447 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:08.664447 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:08.664447 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:08.669953 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 353/692] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:08.675758 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-d1d83a16-24a5-4408-be75-88af24b663e0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 18 12:02:08.675758 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:08.675758 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:08.675758 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:08.675758 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:08.675758 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:08.675758 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:08.675758 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:08.675758 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:08.675758 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:08.675758 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:08.675758 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:08.675758 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 12:02:08.675758 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 12:02:08.675758 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 12:02:08.675758 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 12:02:08.676814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:08.676814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:08.676814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:02:08.676814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:08.676814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:02:08.676814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:08.676814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:08.676814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:08.676814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:08.676814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:08.676814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:08.676814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:08.676814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:08.676814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:08.676814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:08.676814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:08.676814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:08.676814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:08.677769 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:08.677769 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:08.677769 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 12:02:08.677769 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 12:02:08.677769 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 12:02:08.677769 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 12:02:08.677769 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 12:02:08.677769 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 12:02:08.677769 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 18 12:02:08.677769 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:08.677769 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 340/693] 10.222.0.98 () {56 vars in 1096 bytes} [Thu Aug 18 12:02:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:08.682924 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2acaa-46a9-4bd2-a630-7c0d6deafc07 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:08.693566 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d83a16-24a5-4408-be75-88af24b663e0 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:08.723923 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:08.882175 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d83a16-24a5-4408-be75-88af24b663e0 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:08.916338 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2acaa-46a9-4bd2-a630-7c0d6deafc07 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:08.936628 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc322b-df4d-4649-8466-a59195a57874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:08.936908 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc322b-df4d-4649-8466-a59195a57874 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:08.937198 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc322b-df4d-4649-8466-a59195a57874 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:08.937542 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc322b-df4d-4649-8466-a59195a57874 None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:08.938863 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afc322b-df4d-4649-8466-a59195a57874 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:08.939188 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afc322b-df4d-4649-8466-a59195a57874 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:08.940674 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afc322b-df4d-4649-8466-a59195a57874 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:08.960904 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd7a618-eebf-44c9-8543-dfc30f09c55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:08.961819 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7a618-eebf-44c9-8543-dfc30f09c55c None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:08.962249 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7a618-eebf-44c9-8543-dfc30f09c55c None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:08.962726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7a618-eebf-44c9-8543-dfc30f09c55c None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:08.963673 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd7a618-eebf-44c9-8543-dfc30f09c55c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:08.964371 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd7a618-eebf-44c9-8543-dfc30f09c55c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:08.966162 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd7a618-eebf-44c9-8543-dfc30f09c55c None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:08.980583 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 354/694] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:08.982500 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 341/695] 10.222.0.98 () {56 vars in 1135 bytes} [Thu Aug 18 12:02:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:08.988688 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd7a618-eebf-44c9-8543-dfc30f09c55c None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:08.991014 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc322b-df4d-4649-8466-a59195a57874 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:09.072981 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:09.097572 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd7a618-eebf-44c9-8543-dfc30f09c55c None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:09.099816 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:09.136465 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f75295a-58e4-42c3-84f8-15e9a7064d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:09.137354 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75295a-58e4-42c3-84f8-15e9a7064d09 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:09.137814 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75295a-58e4-42c3-84f8-15e9a7064d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:09.138319 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75295a-58e4-42c3-84f8-15e9a7064d09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:09.145913 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f75295a-58e4-42c3-84f8-15e9a7064d09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:09.147772 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f75295a-58e4-42c3-84f8-15e9a7064d09 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:09.171187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-7f75295a-58e4-42c3-84f8-15e9a7064d09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:09.171187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:09.171187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:09.171187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:09.171187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:09.171187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:09.171187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:09.171187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:09.171187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:09.171187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:09.171187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:09.171187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:09.171187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 12:02:09.171187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 12:02:09.171187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 12:02:09.172356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 12:02:09.172356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:09.172356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:09.172356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:09.172356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:09.172356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:09.172356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:09.172356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:09.172356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:09.172356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:09.172356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:09.172356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:09.172356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:09.172356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:09.172356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:09.172356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:09.172356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:09.173801 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 12:02:09.173801 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 12:02:09.173801 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:02:09.173801 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:02:09.173801 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:02:09.173801 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:02:09.173801 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:09.173801 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:09.178982 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 355/696] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:02:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:09.183725 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f75295a-58e4-42c3-84f8-15e9a7064d09 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:09.212356 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc322b-df4d-4649-8466-a59195a57874 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:09.276329 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8213a6-c86e-46f4-affe-d48cc39da8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:09.276958 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8213a6-c86e-46f4-affe-d48cc39da8ee None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:09.277249 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8213a6-c86e-46f4-affe-d48cc39da8ee None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:09.277532 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8213a6-c86e-46f4-affe-d48cc39da8ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:09.304119 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f75295a-58e4-42c3-84f8-15e9a7064d09 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:09.349521 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e239e6-9c7a-4b5e-a4d7-5a384101949a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:09.350736 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e239e6-9c7a-4b5e-a4d7-5a384101949a None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:09.351165 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e239e6-9c7a-4b5e-a4d7-5a384101949a None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:09.352077 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e239e6-9c7a-4b5e-a4d7-5a384101949a None admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:09.353188 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e239e6-9c7a-4b5e-a4d7-5a384101949a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:09.353641 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e239e6-9c7a-4b5e-a4d7-5a384101949a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:09.355849 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e239e6-9c7a-4b5e-a4d7-5a384101949a None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:09.359267 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:09.374731 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 356/697] 10.222.0.98 () {56 vars in 1148 bytes} [Thu Aug 18 12:02:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:09.376382 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8213a6-c86e-46f4-affe-d48cc39da8ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:09.376780 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242f344-1cf7-4e24-9943-a136b1e945b0 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:09.377069 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242f344-1cf7-4e24-9943-a136b1e945b0 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:09.377342 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242f344-1cf7-4e24-9943-a136b1e945b0 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:09.382611 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8213a6-c86e-46f4-affe-d48cc39da8ee None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:09.382771 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 357/698] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:09.384449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948bd881-57e5-42ed-8a12-0b1bde8feaf0 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:09.384734 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948bd881-57e5-42ed-8a12-0b1bde8feaf0 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:09.385009 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948bd881-57e5-42ed-8a12-0b1bde8feaf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:09.405845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-948bd881-57e5-42ed-8a12-0b1bde8feaf0 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:09.407553 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-948bd881-57e5-42ed-8a12-0b1bde8feaf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:09.429821 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:09.487929 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 358/699] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:02:09.489246 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948bd881-57e5-42ed-8a12-0b1bde8feaf0 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:09.507620 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 342/700] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:02:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 18 12:02:09.511778 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8213a6-c86e-46f4-affe-d48cc39da8ee None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:09.628516 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8213a6-c86e-46f4-affe-d48cc39da8ee None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:09.670128 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4203f419-98a6-40f4-a353-b0ab0d006476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:09.670855 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203f419-98a6-40f4-a353-b0ab0d006476 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:09.671133 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203f419-98a6-40f4-a353-b0ab0d006476 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:09.671402 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203f419-98a6-40f4-a353-b0ab0d006476 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:09.680254 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:09.685634 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4203f419-98a6-40f4-a353-b0ab0d006476 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:09.687629 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4203f419-98a6-40f4-a353-b0ab0d006476 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:09.695549 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948bd881-57e5-42ed-8a12-0b1bde8feaf0 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:09.706752 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-4203f419-98a6-40f4-a353-b0ab0d006476 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:09.706752 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:09.706752 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:09.706752 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:09.706752 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:09.706752 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:09.706752 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:09.706752 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:09.706752 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:09.706752 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:09.706752 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:09.706752 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:09.706752 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:09.706752 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:09.706752 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:09.706752 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:09.707812 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:09.707812 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:09.707812 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:09.707812 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:09.707812 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:09.707812 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:09.707812 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:09.707812 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:09.707812 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:09.707812 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:09.707812 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:09.707812 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:09.707812 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:09.707812 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:09.707812 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:09.707812 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:09.707812 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:09.708807 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:09.708807 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:09.708807 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:09.711235 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 343/701] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:09.717407 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4203f419-98a6-40f4-a353-b0ab0d006476 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:09.757979 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:09.780966 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2289bf2d-cd93-4c66-a578-d566082e61a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:09.780966 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289bf2d-cd93-4c66-a578-d566082e61a2 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:09.781906 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289bf2d-cd93-4c66-a578-d566082e61a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:09.782723 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289bf2d-cd93-4c66-a578-d566082e61a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:09.783399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2289bf2d-cd93-4c66-a578-d566082e61a2 None admin] RBAC: Adding query filter params (role.id=c53a0e13b58642f291c856efe1f0b87d, scope.project.id=6f676a5169464d83af4267db25025728, user.id=171c72a6b29e418bbb4ea5c6faffe3b3) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:09.783730 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2289bf2d-cd93-4c66-a578-d566082e61a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:09.785173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2289bf2d-cd93-4c66-a578-d566082e61a2 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:09.799960 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 359/702] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:02:09 2022] GET /identity/v3/role_assignments?user.id=171c72a6b29e418bbb4ea5c6faffe3b3&scope.project.id=6f676a5169464d83af4267db25025728&role.id=c53a0e13b58642f291c856efe1f0b87d => generated 259 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:09.801242 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0942ee2-0de8-4fc3-a296-03a5656b439d None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:09.801529 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0942ee2-0de8-4fc3-a296-03a5656b439d None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:09.801866 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0942ee2-0de8-4fc3-a296-03a5656b439d None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:09.803285 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 360/703] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:09.808579 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf92336-5692-4698-9aea-9f65b871c44c None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:09.808913 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf92336-5692-4698-9aea-9f65b871c44c None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:09.809199 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf92336-5692-4698-9aea-9f65b871c44c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:09.843151 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-fdf92336-5692-4698-9aea-9f65b871c44c None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:09.844433 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-fdf92336-5692-4698-9aea-9f65b871c44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:09.845898 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4203f419-98a6-40f4-a353-b0ab0d006476 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:09.887820 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a79dc-901b-42af-bcde-416e6654ca9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:09.889835 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a79dc-901b-42af-bcde-416e6654ca9b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:09.890325 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a79dc-901b-42af-bcde-416e6654ca9b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:09.890806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a79dc-901b-42af-bcde-416e6654ca9b None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:09.892245 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802a79dc-901b-42af-bcde-416e6654ca9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:09.892843 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802a79dc-901b-42af-bcde-416e6654ca9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:09.894448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802a79dc-901b-42af-bcde-416e6654ca9b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:09.911341 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 344/704] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:09.912661 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a79dc-901b-42af-bcde-416e6654ca9b None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:09.969155 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 361/705] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 12:02:09.970645 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf92336-5692-4698-9aea-9f65b871c44c None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:09.979239 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:10.058520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a79dc-901b-42af-bcde-416e6654ca9b None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:10.095974 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf92336-5692-4698-9aea-9f65b871c44c None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:10.123783 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:10.136259 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314112e1-c91c-416d-b04b-2e2ece61dfdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:10.136911 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314112e1-c91c-416d-b04b-2e2ece61dfdc None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:10.153091 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314112e1-c91c-416d-b04b-2e2ece61dfdc None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:10.153091 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314112e1-c91c-416d-b04b-2e2ece61dfdc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:10.153091 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314112e1-c91c-416d-b04b-2e2ece61dfdc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:10.153091 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314112e1-c91c-416d-b04b-2e2ece61dfdc None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:10.157964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-314112e1-c91c-416d-b04b-2e2ece61dfdc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 12:02:10.157964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:10.157964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 12:02:10.157964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 12:02:10.157964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 12:02:10.157964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 12:02:10.157964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 12:02:10.157964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:10.157964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 12:02:10.157964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:10.157964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 12:02:10.157964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:10.157964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:10.157964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:10.157964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:10.157964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:10.157964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:10.158964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:10.158964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:10.158964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:10.158964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:10.158964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:10.158964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:10.158964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 12:02:10.158964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 12:02:10.158964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 12:02:10.158964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 12:02:10.158964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:10.158964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:10.158964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:10.158964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:10.158964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:10.158964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:10.159844 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:10.159844 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:10.159844 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:10.159844 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:10.159844 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:10.159844 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:10.159844 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:10.159844 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:10.159844 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:10.159844 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:10.159844 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 12:02:10.159844 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:10.159844 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 12:02:10.159844 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:10.159844 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 345/706] 10.222.0.98 () {56 vars in 1129 bytes} [Thu Aug 18 12:02:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:10.164982 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314112e1-c91c-416d-b04b-2e2ece61dfdc None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:10.170528 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf27a7-4d74-40a7-9eef-da64ab01a815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:10.171185 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf27a7-4d74-40a7-9eef-da64ab01a815 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:10.171478 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf27a7-4d74-40a7-9eef-da64ab01a815 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:10.171890 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf27a7-4d74-40a7-9eef-da64ab01a815 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:10.182997 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf27a7-4d74-40a7-9eef-da64ab01a815 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:10.184480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf27a7-4d74-40a7-9eef-da64ab01a815 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:10.209529 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-7aaf27a7-4d74-40a7-9eef-da64ab01a815 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:10.209529 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:10.209529 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:10.209529 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:10.209529 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:10.209529 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:10.209529 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:10.209529 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:10.209529 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:10.209529 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:10.209529 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:10.209529 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:10.209529 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 12:02:10.209529 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 12:02:10.209529 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 12:02:10.211760 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 12:02:10.211760 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:10.211760 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:10.211760 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:10.211760 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:10.211760 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:10.211760 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:10.211760 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:10.211760 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:10.211760 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:10.211760 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:10.211760 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:10.211760 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:10.211760 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:10.211760 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:10.211760 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:10.211760 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:10.214024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 12:02:10.214024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 12:02:10.214024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:02:10.214024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:02:10.214024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:02:10.214024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:02:10.214024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:10.214024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:10.214024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 362/707] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:02:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:10.217436 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf27a7-4d74-40a7-9eef-da64ab01a815 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:10.301883 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314112e1-c91c-416d-b04b-2e2ece61dfdc None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:10.319303 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:10.349928 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11713cdb-94dd-4779-9577-6c267d59ec24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:10.350802 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11713cdb-94dd-4779-9577-6c267d59ec24 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:10.351329 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11713cdb-94dd-4779-9577-6c267d59ec24 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:10.351789 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11713cdb-94dd-4779-9577-6c267d59ec24 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:10.353160 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11713cdb-94dd-4779-9577-6c267d59ec24 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:10.353875 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11713cdb-94dd-4779-9577-6c267d59ec24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:10.356081 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11713cdb-94dd-4779-9577-6c267d59ec24 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:10.363220 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 346/708] 10.222.0.98 () {56 vars in 1124 bytes} [Thu Aug 18 12:02:10 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:10.364227 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11713cdb-94dd-4779-9577-6c267d59ec24 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:10.364502 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf27a7-4d74-40a7-9eef-da64ab01a815 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:10.408088 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8933915d-ff07-46c6-909f-ae8c89a049a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:10.408759 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933915d-ff07-46c6-909f-ae8c89a049a8 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:10.409107 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933915d-ff07-46c6-909f-ae8c89a049a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:10.409396 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933915d-ff07-46c6-909f-ae8c89a049a8 None admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:10.410149 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8933915d-ff07-46c6-909f-ae8c89a049a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:10.410462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8933915d-ff07-46c6-909f-ae8c89a049a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:10.412581 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8933915d-ff07-46c6-909f-ae8c89a049a8 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:10.426706 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 363/709] 10.222.0.98 () {56 vars in 1148 bytes} [Thu Aug 18 12:02:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:10.427692 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8933915d-ff07-46c6-909f-ae8c89a049a8 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:10.473354 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:10.512955 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11713cdb-94dd-4779-9577-6c267d59ec24 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:10.543582 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8933915d-ff07-46c6-909f-ae8c89a049a8 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:10.561891 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0c858-5c66-43a3-82d4-aa83631188f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:10.562535 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0c858-5c66-43a3-82d4-aa83631188f6 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:10.562818 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0c858-5c66-43a3-82d4-aa83631188f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:10.563094 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0c858-5c66-43a3-82d4-aa83631188f6 None admin] PATH_INFO: `/v3/services/network` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:10.563767 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab0c858-5c66-43a3-82d4-aa83631188f6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:10.565155 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab0c858-5c66-43a3-82d4-aa83631188f6 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:10.574712 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-8ab0c858-5c66-43a3-82d4-aa83631188f6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 12:02:10.574712 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:10.574712 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 12:02:10.574712 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 12:02:10.574712 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 12:02:10.574712 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 12:02:10.574712 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 12:02:10.574712 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:10.574712 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 12:02:10.574712 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:10.574712 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 12:02:10.574712 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:10.574712 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:10.574712 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:10.574712 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:10.574712 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:10.574712 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:10.575796 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:10.575796 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:10.575796 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:10.575796 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:10.575796 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:10.575796 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:10.575796 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 12:02:10.575796 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 12:02:10.575796 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 12:02:10.575796 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 12:02:10.575796 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:10.575796 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:10.575796 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:10.575796 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:10.575796 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:10.575796 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:10.576729 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:10.576729 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:10.576729 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:10.576729 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:10.576729 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:10.576729 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:10.576729 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:10.576729 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:10.576729 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:10.576729 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:10.576729 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 12:02:10.576729 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:10.576729 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 12:02:10.576729 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:10.576729 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 347/710] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:02:10 2022] GET /identity/v3/services/network => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:10.577549 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0c858-5c66-43a3-82d4-aa83631188f6 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:10.593323 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8569f33e-0d21-4b9b-a2ff-76e65763aa7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:10.594363 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569f33e-0d21-4b9b-a2ff-76e65763aa7b None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:10.594718 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569f33e-0d21-4b9b-a2ff-76e65763aa7b None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:10.595545 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569f33e-0d21-4b9b-a2ff-76e65763aa7b None admin] PATH_INFO: `/v3/services` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:10.596247 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8569f33e-0d21-4b9b-a2ff-76e65763aa7b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:10.596560 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8569f33e-0d21-4b9b-a2ff-76e65763aa7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:10.598633 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8569f33e-0d21-4b9b-a2ff-76e65763aa7b None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:10.606539 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 364/711] 10.222.0.98 () {56 vars in 1124 bytes} [Thu Aug 18 12:02:10 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:10.607649 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8569f33e-0d21-4b9b-a2ff-76e65763aa7b None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:10.651114 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:10.730527 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0c858-5c66-43a3-82d4-aa83631188f6 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:10.762109 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8569f33e-0d21-4b9b-a2ff-76e65763aa7b None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:10.766758 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:10.778856 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efb8abd-86ef-4840-be6e-4b901cee2f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:10.779512 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb8abd-86ef-4840-be6e-4b901cee2f70 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:10.779796 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb8abd-86ef-4840-be6e-4b901cee2f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:10.780229 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb8abd-86ef-4840-be6e-4b901cee2f70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:10.781351 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efb8abd-86ef-4840-be6e-4b901cee2f70 None admin] RBAC: Adding query filter params (role.id=563e3ed5c6164aae966b64cc302b114a, scope.project.id=6f676a5169464d83af4267db25025728, user.id=771591fe31cb4e8c9c4320c2a7b2719d) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:10.781675 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efb8abd-86ef-4840-be6e-4b901cee2f70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:10.783319 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efb8abd-86ef-4840-be6e-4b901cee2f70 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:10.795443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 348/712] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:02:10 2022] GET /identity/v3/role_assignments?user.id=771591fe31cb4e8c9c4320c2a7b2719d&scope.project.id=6f676a5169464d83af4267db25025728&role.id=563e3ed5c6164aae966b64cc302b114a => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:10.795443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ee29c-08dd-4132-bbfd-5ac00ba12112 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:10.795443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ee29c-08dd-4132-bbfd-5ac00ba12112 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:10.795443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ee29c-08dd-4132-bbfd-5ac00ba12112 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:10.795443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 349/713] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:10.799681 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290dd5e3-0bc3-4387-8dbd-4a66518e011a None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:10.800045 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290dd5e3-0bc3-4387-8dbd-4a66518e011a None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:10.800414 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290dd5e3-0bc3-4387-8dbd-4a66518e011a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:10.811945 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ccf0f-12d2-4ebe-adfd-20e2bb5a512d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:10.812939 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ccf0f-12d2-4ebe-adfd-20e2bb5a512d None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:10.813471 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ccf0f-12d2-4ebe-adfd-20e2bb5a512d None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:10.814434 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ccf0f-12d2-4ebe-adfd-20e2bb5a512d None admin] PATH_INFO: `/v3/services` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:10.815306 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ccf0f-12d2-4ebe-adfd-20e2bb5a512d None admin] RBAC: Adding query filter params (name=network) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:10.815871 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ccf0f-12d2-4ebe-adfd-20e2bb5a512d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:10.818057 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ccf0f-12d2-4ebe-adfd-20e2bb5a512d None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:10.822752 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-290dd5e3-0bc3-4387-8dbd-4a66518e011a None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:10.823818 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 365/714] 10.222.0.98 () {56 vars in 1112 bytes} [Thu Aug 18 12:02:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:10.825257 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-290dd5e3-0bc3-4387-8dbd-4a66518e011a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:10.833546 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ccf0f-12d2-4ebe-adfd-20e2bb5a512d None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:10.948754 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 350/715] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 12:02:10.990499 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ccf0f-12d2-4ebe-adfd-20e2bb5a512d None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:11.027650 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62763720-88a9-4797-81ba-21f5e24b8902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:11.028267 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62763720-88a9-4797-81ba-21f5e24b8902 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:11.028627 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62763720-88a9-4797-81ba-21f5e24b8902 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:11.028917 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62763720-88a9-4797-81ba-21f5e24b8902 None admin] PATH_INFO: `/v3/services` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:11.029552 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62763720-88a9-4797-81ba-21f5e24b8902 None admin] RBAC: Adding query filter params (type=network) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:11.029859 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62763720-88a9-4797-81ba-21f5e24b8902 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:11.031236 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62763720-88a9-4797-81ba-21f5e24b8902 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:11.037282 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 366/716] 10.222.0.98 () {56 vars in 1112 bytes} [Thu Aug 18 12:02:10 2022] GET /identity/v3/services?type=network => generated 124 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:11.242456 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290dd5e3-0bc3-4387-8dbd-4a66518e011a None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:11.278692 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:11.354116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290dd5e3-0bc3-4387-8dbd-4a66518e011a None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:11.396085 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba5eda7-f166-45d1-8567-18cdc217786f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:11.397165 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba5eda7-f166-45d1-8567-18cdc217786f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:11.397682 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba5eda7-f166-45d1-8567-18cdc217786f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:11.398207 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba5eda7-f166-45d1-8567-18cdc217786f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:11.411982 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba5eda7-f166-45d1-8567-18cdc217786f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:11.413829 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba5eda7-f166-45d1-8567-18cdc217786f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:11.436189 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-dba5eda7-f166-45d1-8567-18cdc217786f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:11.436189 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:11.436189 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:11.436189 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:11.436189 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:11.436189 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:11.436189 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:11.436189 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:11.436189 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:11.436189 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:11.436189 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:11.436189 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:11.436189 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:11.436189 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:11.436189 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:11.436189 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:11.437212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:11.437212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:11.437212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:11.437212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:11.437212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:11.437212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:11.437212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:11.437212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:11.437212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:11.437212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:11.437212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:11.437212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:11.437212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:11.437212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:11.437212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:11.437212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:11.437212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:11.438107 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:11.438107 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:11.438107 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:11.439961 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 351/717] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:11.446108 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62763720-88a9-4797-81ba-21f5e24b8902 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:11.497781 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:11.591795 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62763720-88a9-4797-81ba-21f5e24b8902 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:11.641040 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ee6ba-dece-49c8-96f7-17ddaa75c4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:11.642232 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ee6ba-dece-49c8-96f7-17ddaa75c4a2 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:11.642520 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ee6ba-dece-49c8-96f7-17ddaa75c4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:11.642797 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ee6ba-dece-49c8-96f7-17ddaa75c4a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:11.643495 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ee6ba-dece-49c8-96f7-17ddaa75c4a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:11.643781 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ee6ba-dece-49c8-96f7-17ddaa75c4a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:11.645175 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ee6ba-dece-49c8-96f7-17ddaa75c4a2 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:11.666830 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 367/718] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:11.676227 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba5eda7-f166-45d1-8567-18cdc217786f None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:11.831929 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:11.843855 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba5eda7-f166-45d1-8567-18cdc217786f None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:11.893758 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea30b2a-3867-4022-be55-03d984063cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:11.894381 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea30b2a-3867-4022-be55-03d984063cb4 None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:11.894668 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea30b2a-3867-4022-be55-03d984063cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:11.894949 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea30b2a-3867-4022-be55-03d984063cb4 None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:11.895626 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea30b2a-3867-4022-be55-03d984063cb4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:11.897056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea30b2a-3867-4022-be55-03d984063cb4 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:12.009112 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 352/719] 10.222.0.98 () {60 vars in 1132 bytes} [Thu Aug 18 12:02:11 2022] POST /identity/v3/users => generated 254 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:02:12.227884 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3518f48-3e0a-47f4-95a4-e894d3c5e2e3 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:12.228454 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3518f48-3e0a-47f4-95a4-e894d3c5e2e3 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:12.228454 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3518f48-3e0a-47f4-95a4-e894d3c5e2e3 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:12.230247 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 368/720] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:12.241396 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbcccb-5377-4d8d-ba2a-f80bf89f2191 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:12.241727 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbcccb-5377-4d8d-ba2a-f80bf89f2191 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:12.241956 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbcccb-5377-4d8d-ba2a-f80bf89f2191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:12.268200 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-bdfbcccb-5377-4d8d-ba2a-f80bf89f2191 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:12.269603 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-bdfbcccb-5377-4d8d-ba2a-f80bf89f2191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:12.358606 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 353/721] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:02:12.643647 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3518f48-3e0a-47f4-95a4-e894d3c5e2e3 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:12.783618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:12.794371 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3518f48-3e0a-47f4-95a4-e894d3c5e2e3 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:12.838237 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1374c0dc-7dfa-4d4c-be7d-ae043f953237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:12.838878 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374c0dc-7dfa-4d4c-be7d-ae043f953237 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:12.839164 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374c0dc-7dfa-4d4c-be7d-ae043f953237 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:12.839450 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374c0dc-7dfa-4d4c-be7d-ae043f953237 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:12.850858 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1374c0dc-7dfa-4d4c-be7d-ae043f953237 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:02:12.855644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-1374c0dc-7dfa-4d4c-be7d-ae043f953237 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:02:12.855644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:12.855644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:12.855644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:12.855644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:12.855644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:12.855644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:12.855644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:12.855644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:12.855644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:12.855644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:12.855644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:12.855644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:02:12.855644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:02:12.855644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:02:12.855644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:02:12.856602 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:02:12.856602 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:02:12.856602 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:02:12.856602 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:02:12.856602 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:12.856602 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:12.856602 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:12.856602 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:12.856602 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:12.856602 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:12.856602 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:12.856602 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:12.856602 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:12.856602 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:12.856602 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:12.856602 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:12.857446 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:12.857446 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:12.857446 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:12.857446 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:12.857446 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:02:12.857446 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:02:12.857446 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:02:12.857446 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:02:12.857446 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:02:12.857446 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:02:12.857446 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:02:12.857446 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:12.857446 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 369/722] 10.222.0.98 () {56 vars in 1096 bytes} [Thu Aug 18 12:02:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:12.862837 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfbcccb-5377-4d8d-ba2a-f80bf89f2191 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:12.921201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c149a84-8017-449d-89b1-1693bbc48549 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:12.921201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c149a84-8017-449d-89b1-1693bbc48549 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:12.921201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c149a84-8017-449d-89b1-1693bbc48549 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:12.921201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 370/723] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 12:02:12.925913 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0a7bb-6341-4d07-b9f7-6a4ac10002cd None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:12.926213 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0a7bb-6341-4d07-b9f7-6a4ac10002cd None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:12.926499 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0a7bb-6341-4d07-b9f7-6a4ac10002cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:12.962480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-1fa0a7bb-6341-4d07-b9f7-6a4ac10002cd None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:12.963151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:12.963727 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-1fa0a7bb-6341-4d07-b9f7-6a4ac10002cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:13.003522 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfbcccb-5377-4d8d-ba2a-f80bf89f2191 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:13.055140 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 371/724] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 18 12:02:13.090012 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a729bf-5515-40c7-8698-e279b5bb87f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:13.090733 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a729bf-5515-40c7-8698-e279b5bb87f1 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:13.090930 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a729bf-5515-40c7-8698-e279b5bb87f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:13.091180 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a729bf-5515-40c7-8698-e279b5bb87f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:13.091887 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a729bf-5515-40c7-8698-e279b5bb87f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:13.092199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a729bf-5515-40c7-8698-e279b5bb87f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:13.093578 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a729bf-5515-40c7-8698-e279b5bb87f1 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:13.111183 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 354/725] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:02:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:13.116773 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa0a7bb-6341-4d07-b9f7-6a4ac10002cd None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:13.234644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:13.257839 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa0a7bb-6341-4d07-b9f7-6a4ac10002cd None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:13.296368 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2542d642-db39-4e44-b592-c9f6a0c53c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:13.297359 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542d642-db39-4e44-b592-c9f6a0c53c56 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:13.297933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542d642-db39-4e44-b592-c9f6a0c53c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:13.298416 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542d642-db39-4e44-b592-c9f6a0c53c56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:13.301812 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c3ed5-a12b-46cb-a0ef-e54b5b22aa67 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:13.301942 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c3ed5-a12b-46cb-a0ef-e54b5b22aa67 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:13.302152 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c3ed5-a12b-46cb-a0ef-e54b5b22aa67 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:13.303797 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 355/726] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:13.310986 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2542d642-db39-4e44-b592-c9f6a0c53c56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:13.312787 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2542d642-db39-4e44-b592-c9f6a0c53c56 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:13.312917 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4a2ce-37d3-463c-8bb8-d0a785451385 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:13.313125 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4a2ce-37d3-463c-8bb8-d0a785451385 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:13.313407 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4a2ce-37d3-463c-8bb8-d0a785451385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:13.331012 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-2542d642-db39-4e44-b592-c9f6a0c53c56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:13.331012 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:13.331012 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:13.331012 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:13.331012 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:13.331012 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:13.331012 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:13.331012 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:13.331012 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:13.331012 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:13.331012 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:13.331012 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:13.331012 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:13.331012 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:13.331012 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:13.331012 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:13.332044 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:13.332044 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:13.332044 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:13.332044 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:13.332044 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:13.332044 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:13.332044 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:13.332044 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:13.332044 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:13.332044 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:13.332044 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:13.332044 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:13.332044 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:13.332044 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:13.332044 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:13.332044 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:13.332044 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:13.332961 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:13.332961 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:13.332961 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:13.335063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 372/727] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:13.340381 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2542d642-db39-4e44-b592-c9f6a0c53c56 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:13.348434 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-0af4a2ce-37d3-463c-8bb8-d0a785451385 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:13.349651 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-0af4a2ce-37d3-463c-8bb8-d0a785451385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:13.441984 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 356/728] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:02:13.443008 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4a2ce-37d3-463c-8bb8-d0a785451385 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:13.462967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2542d642-db39-4e44-b592-c9f6a0c53c56 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:13.501857 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e09bf8f-6ce1-4bcd-a372-c77ddc0d8790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:13.502845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09bf8f-6ce1-4bcd-a372-c77ddc0d8790 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:13.503334 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09bf8f-6ce1-4bcd-a372-c77ddc0d8790 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:13.503941 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09bf8f-6ce1-4bcd-a372-c77ddc0d8790 None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:13.504913 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e09bf8f-6ce1-4bcd-a372-c77ddc0d8790 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:13.505438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e09bf8f-6ce1-4bcd-a372-c77ddc0d8790 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:13.507114 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e09bf8f-6ce1-4bcd-a372-c77ddc0d8790 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:13.509095 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:13.512244 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:13.523358 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 373/729] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:02:13.531886 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e09bf8f-6ce1-4bcd-a372-c77ddc0d8790 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:13.555291 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4a2ce-37d3-463c-8bb8-d0a785451385 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:13.592165 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5992fa-c5d1-42d2-9fd7-45a3aa788045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:13.593028 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5992fa-c5d1-42d2-9fd7-45a3aa788045 None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:13.593463 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5992fa-c5d1-42d2-9fd7-45a3aa788045 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:13.593923 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5992fa-c5d1-42d2-9fd7-45a3aa788045 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:13.594727 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5992fa-c5d1-42d2-9fd7-45a3aa788045 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:13.596418 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5992fa-c5d1-42d2-9fd7-45a3aa788045 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:13.681585 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e09bf8f-6ce1-4bcd-a372-c77ddc0d8790 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:13.699491 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 357/730] 10.222.0.98 () {60 vars in 1141 bytes} [Thu Aug 18 12:02:13 2022] POST /identity/v3/services => generated 251 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:02:13.701098 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe9f76-1e52-4f5b-b538-439688efc525 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:13.701570 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe9f76-1e52-4f5b-b538-439688efc525 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:13.702207 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe9f76-1e52-4f5b-b538-439688efc525 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:13.703908 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 358/731] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:13.712942 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09402f8a-1da8-488c-b82b-4ba837308001 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:13.714723 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09402f8a-1da8-488c-b82b-4ba837308001 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:13.716225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09402f8a-1da8-488c-b82b-4ba837308001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:13.757893 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-09402f8a-1da8-488c-b82b-4ba837308001 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:13.759911 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-09402f8a-1da8-488c-b82b-4ba837308001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:13.770810 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d9311-adb2-4685-a029-b733fa27dc86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:13.773415 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d9311-adb2-4685-a029-b733fa27dc86 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:13.774103 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d9311-adb2-4685-a029-b733fa27dc86 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:13.777139 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d9311-adb2-4685-a029-b733fa27dc86 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:13.791126 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277d9311-adb2-4685-a029-b733fa27dc86 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:13.792662 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277d9311-adb2-4685-a029-b733fa27dc86 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:13.811939 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-277d9311-adb2-4685-a029-b733fa27dc86 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 12:02:13.811939 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:13.811939 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:13.811939 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:13.811939 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:13.811939 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:13.811939 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:13.811939 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:13.811939 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:13.811939 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:13.811939 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:13.811939 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:13.811939 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 12:02:13.811939 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 12:02:13.811939 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 12:02:13.811939 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 12:02:13.812965 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:13.812965 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:13.812965 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:02:13.812965 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:13.812965 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:02:13.812965 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:13.812965 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:13.812965 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:13.812965 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:13.812965 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:13.812965 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:13.812965 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:13.812965 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:13.812965 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:13.812965 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:13.812965 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:13.812965 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:13.812965 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:13.814722 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:13.814722 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:13.814722 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 12:02:13.814722 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 12:02:13.814722 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 12:02:13.814722 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 12:02:13.814722 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 12:02:13.814722 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 12:02:13.814722 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 12:02:13.814722 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:13.817987 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 374/732] 10.222.0.98 () {56 vars in 1093 bytes} [Thu Aug 18 12:02:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:13.819093 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d9311-adb2-4685-a029-b733fa27dc86 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:13.865836 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 359/733] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:13 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 18 12:02:13.871937 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09402f8a-1da8-488c-b82b-4ba837308001 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:13.881048 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:13.942266 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d9311-adb2-4685-a029-b733fa27dc86 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:13.944752 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:13.978931 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2db4e7-876f-40bb-b454-9a5f8dde7ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:13.979881 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2db4e7-876f-40bb-b454-9a5f8dde7ebb None admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:13.980106 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2db4e7-876f-40bb-b454-9a5f8dde7ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:13.980482 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2db4e7-876f-40bb-b454-9a5f8dde7ebb None admin] PATH_INFO: `/v3/services` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:13.981198 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2db4e7-876f-40bb-b454-9a5f8dde7ebb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:13.982659 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2db4e7-876f-40bb-b454-9a5f8dde7ebb None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:14.003246 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09402f8a-1da8-488c-b82b-4ba837308001 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:14.042532 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd100de-f3a3-4c7f-bf47-51f71a342077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:14.043569 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd100de-f3a3-4c7f-bf47-51f71a342077 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:14.044005 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd100de-f3a3-4c7f-bf47-51f71a342077 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:14.044431 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd100de-f3a3-4c7f-bf47-51f71a342077 None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:14.045339 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd100de-f3a3-4c7f-bf47-51f71a342077 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:14.045971 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd100de-f3a3-4c7f-bf47-51f71a342077 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:14.047686 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd100de-f3a3-4c7f-bf47-51f71a342077 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:14.071963 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 360/734] 10.222.0.98 () {56 vars in 1133 bytes} [Thu Aug 18 12:02:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:14.073788 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd100de-f3a3-4c7f-bf47-51f71a342077 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:14.084437 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 375/735] 10.222.0.98 () {60 vars in 1141 bytes} [Thu Aug 18 12:02:13 2022] POST /identity/v3/services => generated 240 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:02:14.207667 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2db4e7-876f-40bb-b454-9a5f8dde7ebb None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:14.234479 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd100de-f3a3-4c7f-bf47-51f71a342077 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:14.256804 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:14.307910 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea6590-1d30-4d6f-85ea-329fd92552e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:14.308475 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea6590-1d30-4d6f-85ea-329fd92552e9 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:14.308756 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea6590-1d30-4d6f-85ea-329fd92552e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:14.309028 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea6590-1d30-4d6f-85ea-329fd92552e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:14.323848 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea6590-1d30-4d6f-85ea-329fd92552e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:14.325409 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea6590-1d30-4d6f-85ea-329fd92552e9 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:14.340811 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2db4e7-876f-40bb-b454-9a5f8dde7ebb None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:14.345001 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-87ea6590-1d30-4d6f-85ea-329fd92552e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:14.345001 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:14.345001 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:14.345001 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:14.345001 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:14.345001 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:14.345001 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:14.345001 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:14.345001 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:14.345001 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:14.345001 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:14.345001 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:14.345001 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:14.345001 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:14.345001 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:14.345001 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:14.345949 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:14.345949 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:14.345949 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:14.345949 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:14.345949 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:14.345949 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:14.345949 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:14.345949 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:14.345949 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:14.345949 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:14.345949 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:14.345949 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:14.345949 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:14.345949 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:14.345949 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:14.345949 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:14.345949 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:14.346849 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:14.346849 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:14.346849 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:14.346849 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 361/736] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:02:14.354013 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea6590-1d30-4d6f-85ea-329fd92552e9 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:14.356310 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:14.384935 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65560d5a-96c3-4ff7-8226-f6b0105151b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:14.385584 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65560d5a-96c3-4ff7-8226-f6b0105151b7 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:14.385868 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65560d5a-96c3-4ff7-8226-f6b0105151b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:14.386149 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65560d5a-96c3-4ff7-8226-f6b0105151b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:14.401632 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65560d5a-96c3-4ff7-8226-f6b0105151b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:02:14.406261 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-65560d5a-96c3-4ff7-8226-f6b0105151b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:02:14.406261 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:14.406261 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:14.406261 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:14.406261 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:14.406261 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:14.406261 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:14.406261 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:14.406261 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:14.406261 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:14.406261 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:14.406261 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:14.406261 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:02:14.406261 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:02:14.406261 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:02:14.406261 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:02:14.407350 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:02:14.407350 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:02:14.407350 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:02:14.407350 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:02:14.407350 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:14.407350 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:14.407350 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:14.407350 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:14.407350 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:14.407350 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:14.407350 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:14.407350 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:14.407350 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:14.407350 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:14.407350 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:14.407350 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:14.409029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:14.409029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:14.409029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:14.409029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:14.409029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:02:14.409029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:02:14.409029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:02:14.409029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:02:14.409029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:02:14.409029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:02:14.409029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:02:14.409029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:14.409029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 376/737] 10.222.0.98 () {56 vars in 1096 bytes} [Thu Aug 18 12:02:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:02:14.414214 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65560d5a-96c3-4ff7-8226-f6b0105151b7 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:14.493633 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea6590-1d30-4d6f-85ea-329fd92552e9 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:14.540929 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd75949-4bd5-4145-9aac-8a78a74b8896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:14.543118 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd75949-4bd5-4145-9aac-8a78a74b8896 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:14.543873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd75949-4bd5-4145-9aac-8a78a74b8896 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:14.545412 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd75949-4bd5-4145-9aac-8a78a74b8896 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:14.546388 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd75949-4bd5-4145-9aac-8a78a74b8896 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:14.547881 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd75949-4bd5-4145-9aac-8a78a74b8896 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:14.551596 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd75949-4bd5-4145-9aac-8a78a74b8896 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:14.578281 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 362/738] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:02:14.601013 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd75949-4bd5-4145-9aac-8a78a74b8896 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:14.633350 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65560d5a-96c3-4ff7-8226-f6b0105151b7 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:14.635864 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:14.702457 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2722b1-f15a-412f-9d0a-012cc3252dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:14.703510 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2722b1-f15a-412f-9d0a-012cc3252dcb None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:14.704011 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2722b1-f15a-412f-9d0a-012cc3252dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:14.704538 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2722b1-f15a-412f-9d0a-012cc3252dcb None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:14.705403 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2722b1-f15a-412f-9d0a-012cc3252dcb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:14.705905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2722b1-f15a-412f-9d0a-012cc3252dcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:14.707651 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2722b1-f15a-412f-9d0a-012cc3252dcb None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:14.717330 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 377/739] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:02:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:14.719010 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c80de3-4c3a-4f9f-ad68-a4ea9f02c68e None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:14.719550 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c80de3-4c3a-4f9f-ad68-a4ea9f02c68e None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:14.720635 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c80de3-4c3a-4f9f-ad68-a4ea9f02c68e None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:14.722487 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 378/740] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:14.723409 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c80de3-4c3a-4f9f-ad68-a4ea9f02c68e None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:14.728854 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd75949-4bd5-4145-9aac-8a78a74b8896 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:14.759562 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:14.782037 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f123dc36-deb9-4569-bf85-b76f9e850565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:14.783692 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f123dc36-deb9-4569-bf85-b76f9e850565 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:14.784442 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f123dc36-deb9-4569-bf85-b76f9e850565 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:14.784910 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f123dc36-deb9-4569-bf85-b76f9e850565 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:14.796361 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f123dc36-deb9-4569-bf85-b76f9e850565 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:14.799437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f123dc36-deb9-4569-bf85-b76f9e850565 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:14.815522 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-f123dc36-deb9-4569-bf85-b76f9e850565 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:14.815522 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:14.815522 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:14.815522 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:14.815522 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:14.815522 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:14.815522 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:14.815522 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:14.815522 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:14.815522 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:14.815522 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:14.815522 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:14.815522 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 12:02:14.815522 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 12:02:14.815522 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 12:02:14.816503 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 12:02:14.816503 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:14.816503 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:14.816503 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:14.816503 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:14.816503 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:14.816503 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:14.816503 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:14.816503 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:14.816503 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:14.816503 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:14.816503 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:14.816503 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:14.816503 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:14.816503 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:14.816503 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:14.816503 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:14.817401 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 12:02:14.817401 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 12:02:14.817401 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:02:14.817401 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:02:14.817401 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:02:14.817401 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:02:14.817401 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:14.817401 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:14.824669 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 363/741] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:02:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:02:14.825360 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f35cb37-3e89-49c3-9471-125e34118fc8 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:14.825649 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f35cb37-3e89-49c3-9471-125e34118fc8 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:14.826025 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f35cb37-3e89-49c3-9471-125e34118fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:14.844811 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-5f35cb37-3e89-49c3-9471-125e34118fc8 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:14.846459 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-5f35cb37-3e89-49c3-9471-125e34118fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:14.853812 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c80de3-4c3a-4f9f-ad68-a4ea9f02c68e None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:14.904797 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209346f4-1ada-4a74-9ae4-356ca7d4debe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:14.905832 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209346f4-1ada-4a74-9ae4-356ca7d4debe None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:14.906326 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209346f4-1ada-4a74-9ae4-356ca7d4debe None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:14.906815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209346f4-1ada-4a74-9ae4-356ca7d4debe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:14.924015 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209346f4-1ada-4a74-9ae4-356ca7d4debe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:14.925824 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209346f4-1ada-4a74-9ae4-356ca7d4debe None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:14.927502 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 364/742] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:02:14.928615 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f35cb37-3e89-49c3-9471-125e34118fc8 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:14.962147 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-209346f4-1ada-4a74-9ae4-356ca7d4debe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:14.962147 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:14.962147 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:14.962147 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:14.962147 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:14.962147 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:14.962147 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:14.962147 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:14.962147 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:14.962147 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:14.962147 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:14.962147 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:14.962147 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:14.962147 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:14.962147 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:14.962147 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:14.963438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:14.963438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:14.963438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:14.963438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:14.963438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:14.963438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:14.963438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:14.963438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:14.963438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:14.963438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:14.963438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:14.963438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:14.963438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:14.963438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:14.963438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:14.963438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:14.963438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:14.964341 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:14.964341 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:14.964341 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:14.969587 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:14.972716 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 379/743] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:14.976397 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209346f4-1ada-4a74-9ae4-356ca7d4debe None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:15.107180 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f35cb37-3e89-49c3-9471-125e34118fc8 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:15.144788 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526e6dd-b1bb-41ab-aa9d-511d67ab19e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:15.145836 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526e6dd-b1bb-41ab-aa9d-511d67ab19e4 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:15.146326 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526e6dd-b1bb-41ab-aa9d-511d67ab19e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:15.146806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526e6dd-b1bb-41ab-aa9d-511d67ab19e4 None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:15.147805 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:15.148321 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f526e6dd-b1bb-41ab-aa9d-511d67ab19e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:15.149538 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f526e6dd-b1bb-41ab-aa9d-511d67ab19e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:15.152220 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f526e6dd-b1bb-41ab-aa9d-511d67ab19e4 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:15.163829 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209346f4-1ada-4a74-9ae4-356ca7d4debe None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:15.165356 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 365/744] 10.222.0.98 () {56 vars in 1148 bytes} [Thu Aug 18 12:02:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:15.171516 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526e6dd-b1bb-41ab-aa9d-511d67ab19e4 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:15.199623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e67f67-dccc-4b48-9bdb-7b9a0dc345e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:15.200234 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e67f67-dccc-4b48-9bdb-7b9a0dc345e4 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:15.200562 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e67f67-dccc-4b48-9bdb-7b9a0dc345e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:15.200835 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e67f67-dccc-4b48-9bdb-7b9a0dc345e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:15.201578 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e67f67-dccc-4b48-9bdb-7b9a0dc345e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:15.201826 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e67f67-dccc-4b48-9bdb-7b9a0dc345e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:15.203177 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e67f67-dccc-4b48-9bdb-7b9a0dc345e4 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:15.216790 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 380/745] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:15.222550 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e67f67-dccc-4b48-9bdb-7b9a0dc345e4 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:15.275577 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:15.307556 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526e6dd-b1bb-41ab-aa9d-511d67ab19e4 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:15.331407 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e67f67-dccc-4b48-9bdb-7b9a0dc345e4 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:15.345776 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9f5e02-5b05-426b-91ef-f574b93c6235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:15.346553 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f5e02-5b05-426b-91ef-f574b93c6235 None admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:15.346845 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f5e02-5b05-426b-91ef-f574b93c6235 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:15.347120 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f5e02-5b05-426b-91ef-f574b93c6235 None admin] PATH_INFO: `/v3/projects/6f676a5169464d83af4267db25025728/users/171c72a6b29e418bbb4ea5c6faffe3b3/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:15.349278 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9f5e02-5b05-426b-91ef-f574b93c6235 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f676a5169464d83af4267db25025728, user_id=171c72a6b29e418bbb4ea5c6faffe3b3, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:15.351026 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9f5e02-5b05-426b-91ef-f574b93c6235 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:15.370863 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef61f31-0835-47e7-a9d5-44e63fc2d839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:15.370863 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef61f31-0835-47e7-a9d5-44e63fc2d839 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:15.370863 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef61f31-0835-47e7-a9d5-44e63fc2d839 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:15.370863 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef61f31-0835-47e7-a9d5-44e63fc2d839 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:15.378642 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef61f31-0835-47e7-a9d5-44e63fc2d839 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:15.380217 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef61f31-0835-47e7-a9d5-44e63fc2d839 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:15.392387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-8ef61f31-0835-47e7-a9d5-44e63fc2d839 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 18 12:02:15.392387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:15.392387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:15.392387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:15.392387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:15.392387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:15.392387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:15.392387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:15.392387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:15.392387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:15.392387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:15.392387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:15.392387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 12:02:15.392387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 12:02:15.392387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 12:02:15.392387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 12:02:15.393432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:15.393432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:15.393432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:02:15.393432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:15.393432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:02:15.393432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:15.393432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:15.393432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:15.393432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:15.393432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:15.393432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:15.393432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:15.393432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:15.393432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:15.393432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:15.393432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:15.393432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:15.393432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:15.394490 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:15.394490 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:15.394490 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 12:02:15.394490 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 12:02:15.394490 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 12:02:15.394490 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 12:02:15.394490 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 12:02:15.394490 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 12:02:15.394490 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 18 12:02:15.394490 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:15.398156 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 381/746] 10.222.0.98 () {56 vars in 1096 bytes} [Thu Aug 18 12:02:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:02:15.398998 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef61f31-0835-47e7-a9d5-44e63fc2d839 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:15.403976 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:15.429067 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 366/747] 10.222.0.98 () {58 vars in 1439 bytes} [Thu Aug 18 12:02:15 2022] PUT /identity/v3/projects/6f676a5169464d83af4267db25025728/users/171c72a6b29e418bbb4ea5c6faffe3b3/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:02:15.430473 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9f5e02-5b05-426b-91ef-f574b93c6235 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:15.564366 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9f5e02-5b05-426b-91ef-f574b93c6235 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:15.602229 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:15.604049 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef61f31-0835-47e7-a9d5-44e63fc2d839 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:15.608799 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50878cf3-5980-443d-b1d6-2a610117eb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:15.609802 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50878cf3-5980-443d-b1d6-2a610117eb35 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:15.611407 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50878cf3-5980-443d-b1d6-2a610117eb35 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:15.611798 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50878cf3-5980-443d-b1d6-2a610117eb35 None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:15.613893 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50878cf3-5980-443d-b1d6-2a610117eb35 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:15.614429 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50878cf3-5980-443d-b1d6-2a610117eb35 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:15.616226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50878cf3-5980-443d-b1d6-2a610117eb35 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:15.641210 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 367/748] 10.222.0.98 () {56 vars in 1135 bytes} [Thu Aug 18 12:02:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:15.649741 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50878cf3-5980-443d-b1d6-2a610117eb35 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:15.697149 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee9134-a975-4735-b1c8-32cb481e189f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:15.697493 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee9134-a975-4735-b1c8-32cb481e189f None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:15.697777 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee9134-a975-4735-b1c8-32cb481e189f None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:15.698048 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee9134-a975-4735-b1c8-32cb481e189f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:15.715802 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee9134-a975-4735-b1c8-32cb481e189f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:02:15.723165 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-5bee9134-a975-4735-b1c8-32cb481e189f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:02:15.723165 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:15.723165 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:15.723165 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:15.723165 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:15.723165 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:15.723165 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:15.723165 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:15.723165 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:15.723165 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:15.723165 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:15.723165 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:15.723165 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:02:15.723165 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:02:15.723165 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:02:15.723165 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:02:15.724226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:02:15.724226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:02:15.724226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:02:15.724226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:02:15.724226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:15.724226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:15.724226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:15.724226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:15.724226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:15.724226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:15.724226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:15.724226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:15.724226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:15.724226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:15.724226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:15.724226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:15.725063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:15.725063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:15.725063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:15.725063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:15.725063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:02:15.725063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:02:15.725063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:02:15.725063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:02:15.725063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:02:15.725063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:02:15.725063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:02:15.725063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:15.725063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 382/749] 10.222.0.98 () {56 vars in 1102 bytes} [Thu Aug 18 12:02:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:15.731754 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee9134-a975-4735-b1c8-32cb481e189f None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:15.792362 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50878cf3-5980-443d-b1d6-2a610117eb35 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:15.815999 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:15.843565 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b93891-4fe7-4dfa-9ee1-ca74fbdfb1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:15.844813 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b93891-4fe7-4dfa-9ee1-ca74fbdfb1e8 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:15.845938 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b93891-4fe7-4dfa-9ee1-ca74fbdfb1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:15.846672 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b93891-4fe7-4dfa-9ee1-ca74fbdfb1e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:15.866043 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee9134-a975-4735-b1c8-32cb481e189f None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:15.926995 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:15.939123 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b93891-4fe7-4dfa-9ee1-ca74fbdfb1e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:15.941176 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b93891-4fe7-4dfa-9ee1-ca74fbdfb1e8 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:15.949390 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed9738-cc70-4897-bba1-873dfd9dee80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:15.950014 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed9738-cc70-4897-bba1-873dfd9dee80 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:15.950302 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed9738-cc70-4897-bba1-873dfd9dee80 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:15.950633 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed9738-cc70-4897-bba1-873dfd9dee80 None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:15.951302 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed9738-cc70-4897-bba1-873dfd9dee80 None admin] RBAC: Adding query filter params (name=service) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:15.951628 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed9738-cc70-4897-bba1-873dfd9dee80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:15.953035 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed9738-cc70-4897-bba1-873dfd9dee80 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:15.970910 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 383/750] 10.222.0.98 () {56 vars in 1103 bytes} [Thu Aug 18 12:02:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:02:15.980504 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed9738-cc70-4897-bba1-873dfd9dee80 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:16.062253 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 368/751] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:02:15 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 12:02:16.066955 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732cce60-1d0e-4d93-93a8-7697df44b069 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:16.066955 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732cce60-1d0e-4d93-93a8-7697df44b069 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:16.066955 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732cce60-1d0e-4d93-93a8-7697df44b069 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:16.066955 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 369/752] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:16.069563 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732cce60-1d0e-4d93-93a8-7697df44b069 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:16.122019 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed9738-cc70-4897-bba1-873dfd9dee80 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:16.161328 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:16.175485 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51766f05-fda1-4be3-bd35-2ecfd7703ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:16.177164 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51766f05-fda1-4be3-bd35-2ecfd7703ff9 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:16.177638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51766f05-fda1-4be3-bd35-2ecfd7703ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:16.178370 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51766f05-fda1-4be3-bd35-2ecfd7703ff9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:16.179879 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:16.191268 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732cce60-1d0e-4d93-93a8-7697df44b069 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:16.195633 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51766f05-fda1-4be3-bd35-2ecfd7703ff9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:16.197357 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51766f05-fda1-4be3-bd35-2ecfd7703ff9 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:16.217508 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-51766f05-fda1-4be3-bd35-2ecfd7703ff9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:16.217508 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:16.217508 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:16.217508 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:16.217508 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:16.217508 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:16.217508 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:16.217508 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:16.217508 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:16.217508 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:16.217508 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:16.217508 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:16.217508 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:16.217508 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:16.217508 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:16.217508 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:16.218524 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:16.218524 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:16.218524 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:16.218524 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:16.218524 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:16.218524 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:16.218524 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:16.218524 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:16.218524 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:16.218524 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:16.218524 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:16.218524 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:16.218524 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:16.218524 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:16.218524 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:16.218524 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:16.218524 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:16.219481 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:16.219481 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:16.219481 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:16.226424 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 384/753] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:16.226424 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d3c75-97dc-4cb7-bdeb-5969b3323676 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:16.226424 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d3c75-97dc-4cb7-bdeb-5969b3323676 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:16.226424 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d3c75-97dc-4cb7-bdeb-5969b3323676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:16.245165 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7818e51a-7e0a-43b4-996b-029b1c238259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:16.246274 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7818e51a-7e0a-43b4-996b-029b1c238259 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:16.246763 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7818e51a-7e0a-43b4-996b-029b1c238259 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:16.247275 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7818e51a-7e0a-43b4-996b-029b1c238259 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:16.251150 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-d76d3c75-97dc-4cb7-bdeb-5969b3323676 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:16.252951 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-d76d3c75-97dc-4cb7-bdeb-5969b3323676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:16.261549 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7818e51a-7e0a-43b4-996b-029b1c238259 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:16.263479 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7818e51a-7e0a-43b4-996b-029b1c238259 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:16.280660 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-7818e51a-7e0a-43b4-996b-029b1c238259 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:16.280660 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:16.280660 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:16.280660 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:16.280660 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:16.280660 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:16.280660 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:16.280660 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:16.280660 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:16.280660 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:16.280660 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:16.280660 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:16.280660 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:16.280660 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:16.280660 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:16.280660 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:16.282093 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:16.282093 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:16.282093 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:16.282093 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:16.282093 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:16.282093 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:16.282093 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:16.282093 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:16.282093 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:16.282093 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:16.282093 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:16.282093 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:16.282093 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:16.282093 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:16.282093 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:16.282093 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:16.282093 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:16.283559 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:16.283559 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:16.283559 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:16.286028 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 370/754] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:16.287171 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7818e51a-7e0a-43b4-996b-029b1c238259 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:16.339460 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 385/755] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:02:16.339460 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d3c75-97dc-4cb7-bdeb-5969b3323676 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:16.402852 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7818e51a-7e0a-43b4-996b-029b1c238259 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:16.440916 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43e013d-5f7b-42ec-9bc0-f3934ed645de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:16.441431 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e013d-5f7b-42ec-9bc0-f3934ed645de None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:16.441717 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e013d-5f7b-42ec-9bc0-f3934ed645de None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:16.441994 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e013d-5f7b-42ec-9bc0-f3934ed645de None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:16.442765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43e013d-5f7b-42ec-9bc0-f3934ed645de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:16.443077 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43e013d-5f7b-42ec-9bc0-f3934ed645de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:16.444485 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43e013d-5f7b-42ec-9bc0-f3934ed645de None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:16.458259 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 371/756] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:16.467438 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260dd4e2-9845-4e7e-b9c2-22bf46f93d68 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:16.467438 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260dd4e2-9845-4e7e-b9c2-22bf46f93d68 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:16.467438 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260dd4e2-9845-4e7e-b9c2-22bf46f93d68 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:16.467438 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 372/757] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:16.467438 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260dd4e2-9845-4e7e-b9c2-22bf46f93d68 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:16.469385 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:16.478493 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d3c75-97dc-4cb7-bdeb-5969b3323676 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:16.492317 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:16.525912 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac01516-fb3d-433d-b514-a8ffe71c5d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:16.526788 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac01516-fb3d-433d-b514-a8ffe71c5d4b None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:16.527088 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac01516-fb3d-433d-b514-a8ffe71c5d4b None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:16.527771 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac01516-fb3d-433d-b514-a8ffe71c5d4b None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:16.528796 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac01516-fb3d-433d-b514-a8ffe71c5d4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:16.529320 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac01516-fb3d-433d-b514-a8ffe71c5d4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:16.531998 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac01516-fb3d-433d-b514-a8ffe71c5d4b None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:16.551449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 386/758] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:16.551449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5741d86-e8d9-4409-ae7f-63410227adb6 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:16.551449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5741d86-e8d9-4409-ae7f-63410227adb6 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:16.551449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5741d86-e8d9-4409-ae7f-63410227adb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:16.574330 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-f5741d86-e8d9-4409-ae7f-63410227adb6 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:16.575670 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-f5741d86-e8d9-4409-ae7f-63410227adb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:16.592266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260dd4e2-9845-4e7e-b9c2-22bf46f93d68 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:16.636794 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee00c1-c24d-4c28-bfa8-cabe95ade6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:16.637949 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee00c1-c24d-4c28-bfa8-cabe95ade6af None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:16.638841 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee00c1-c24d-4c28-bfa8-cabe95ade6af None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:16.639528 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee00c1-c24d-4c28-bfa8-cabe95ade6af None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:16.649503 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee00c1-c24d-4c28-bfa8-cabe95ade6af None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:16.652569 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee00c1-c24d-4c28-bfa8-cabe95ade6af None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:16.659139 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 387/759] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:02:16.660010 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5741d86-e8d9-4409-ae7f-63410227adb6 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:16.669714 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-b3ee00c1-c24d-4c28-bfa8-cabe95ade6af None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 12:02:16.669714 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:16.669714 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:16.669714 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:16.669714 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:16.669714 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:16.669714 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:16.669714 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:16.669714 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:16.669714 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:16.669714 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:16.669714 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:16.669714 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 12:02:16.669714 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 12:02:16.669714 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 12:02:16.669714 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 12:02:16.670700 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:16.670700 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:16.670700 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:02:16.670700 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:16.670700 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:02:16.670700 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:16.670700 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:16.670700 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:16.670700 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:16.670700 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:16.670700 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:16.670700 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:16.670700 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:16.670700 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:16.670700 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:16.670700 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:16.670700 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:16.670700 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:16.671712 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:16.671712 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:16.671712 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 12:02:16.671712 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 12:02:16.671712 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 12:02:16.671712 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 12:02:16.671712 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 12:02:16.671712 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 12:02:16.671712 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 12:02:16.671712 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:16.674893 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 373/760] 10.222.0.98 () {56 vars in 1117 bytes} [Thu Aug 18 12:02:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:16.676064 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee00c1-c24d-4c28-bfa8-cabe95ade6af None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:16.768549 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:16.781000 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:16.785362 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5741d86-e8d9-4409-ae7f-63410227adb6 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:16.794558 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee00c1-c24d-4c28-bfa8-cabe95ade6af None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:16.822674 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4747b028-f739-4bdf-9793-cd26735cb946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:16.823377 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747b028-f739-4bdf-9793-cd26735cb946 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:16.823794 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747b028-f739-4bdf-9793-cd26735cb946 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:16.828330 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747b028-f739-4bdf-9793-cd26735cb946 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:16.831749 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d079d9-0d30-4f79-b993-8ad31c0a3ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:16.831889 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4747b028-f739-4bdf-9793-cd26735cb946 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:16.832687 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d079d9-0d30-4f79-b993-8ad31c0a3ee8 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:16.833133 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d079d9-0d30-4f79-b993-8ad31c0a3ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:16.833558 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4747b028-f739-4bdf-9793-cd26735cb946 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:16.833670 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d079d9-0d30-4f79-b993-8ad31c0a3ee8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:16.834376 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d079d9-0d30-4f79-b993-8ad31c0a3ee8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:16.836024 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d079d9-0d30-4f79-b993-8ad31c0a3ee8 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:16.847514 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-4747b028-f739-4bdf-9793-cd26735cb946 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:16.847514 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:16.847514 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:16.847514 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:16.847514 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:16.847514 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:16.847514 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:16.847514 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:16.847514 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:16.847514 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:16.847514 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:16.847514 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:16.847514 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 12:02:16.847514 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 12:02:16.847514 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 12:02:16.848593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 12:02:16.848593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:16.848593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:16.848593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:16.848593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:16.848593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:16.848593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:16.848593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:16.848593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:16.848593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:16.848593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:16.848593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:16.848593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:16.848593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:16.848593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:16.848593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:16.848593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:16.849518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 12:02:16.849518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 12:02:16.849518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:02:16.849518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:02:16.849518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:02:16.849518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:02:16.849518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:16.849518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:16.850454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-89d079d9-0d30-4f79-b993-8ad31c0a3ee8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 12:02:16.850454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:16.850454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 12:02:16.850454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 12:02:16.850454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 12:02:16.850454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 12:02:16.850454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 12:02:16.850454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:16.850454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 12:02:16.850454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:16.850454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 12:02:16.850454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:16.850454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:16.850454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:16.850454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:16.850454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:16.850454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:16.851390 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:16.851390 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:16.851390 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:16.851390 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:16.851390 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:16.851390 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:16.851390 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 12:02:16.851390 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 12:02:16.851390 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 12:02:16.851390 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 12:02:16.851390 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:16.851390 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:16.851390 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:16.851390 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:16.851390 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:16.851390 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:16.852339 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:16.852339 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:16.852339 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:16.852339 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:16.852339 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:16.852339 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:16.852339 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:16.852339 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:16.852339 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:16.852339 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:16.852339 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 12:02:16.852339 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:16.852339 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 12:02:16.852339 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:16.853072 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 388/761] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:02:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:16.853609 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4747b028-f739-4bdf-9793-cd26735cb946 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:16.854662 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 374/762] 10.222.0.98 () {56 vars in 1129 bytes} [Thu Aug 18 12:02:16 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:16.855616 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d079d9-0d30-4f79-b993-8ad31c0a3ee8 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:16.978919 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4747b028-f739-4bdf-9793-cd26735cb946 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:16.985548 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d079d9-0d30-4f79-b993-8ad31c0a3ee8 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:17.015783 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4209cd69-f2e6-4101-84e2-0b6e3ef72b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:17.016799 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209cd69-f2e6-4101-84e2-0b6e3ef72b5e None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:17.017653 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209cd69-f2e6-4101-84e2-0b6e3ef72b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:17.018192 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209cd69-f2e6-4101-84e2-0b6e3ef72b5e None admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:17.018925 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4209cd69-f2e6-4101-84e2-0b6e3ef72b5e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:17.019440 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:17.020722 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4209cd69-f2e6-4101-84e2-0b6e3ef72b5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:17.022412 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4209cd69-f2e6-4101-84e2-0b6e3ef72b5e None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:17.029130 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf4406f-2432-4fa2-a0d5-0055c7b4f694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:17.029742 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4406f-2432-4fa2-a0d5-0055c7b4f694 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:17.030051 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4406f-2432-4fa2-a0d5-0055c7b4f694 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:17.030339 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4406f-2432-4fa2-a0d5-0055c7b4f694 None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:17.030985 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf4406f-2432-4fa2-a0d5-0055c7b4f694 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:17.031394 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf4406f-2432-4fa2-a0d5-0055c7b4f694 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:17.033470 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf4406f-2432-4fa2-a0d5-0055c7b4f694 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:17.041699 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 389/763] 10.222.0.98 () {56 vars in 1149 bytes} [Thu Aug 18 12:02:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:17.042348 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4209cd69-f2e6-4101-84e2-0b6e3ef72b5e None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:17.054573 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 375/764] 10.222.0.98 () {56 vars in 1148 bytes} [Thu Aug 18 12:02:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:17.055460 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf4406f-2432-4fa2-a0d5-0055c7b4f694 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:17.061372 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:17.150896 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4209cd69-f2e6-4101-84e2-0b6e3ef72b5e None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:17.169112 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf4406f-2432-4fa2-a0d5-0055c7b4f694 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:17.188550 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358739e-e1a5-4e2c-8c90-5c54b90f37d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:17.189208 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358739e-e1a5-4e2c-8c90-5c54b90f37d5 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:17.189490 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358739e-e1a5-4e2c-8c90-5c54b90f37d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:17.189762 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358739e-e1a5-4e2c-8c90-5c54b90f37d5 None admin] PATH_INFO: `/v3/services` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:17.190395 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358739e-e1a5-4e2c-8c90-5c54b90f37d5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:17.190702 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358739e-e1a5-4e2c-8c90-5c54b90f37d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:17.192233 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358739e-e1a5-4e2c-8c90-5c54b90f37d5 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:17.197838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 390/765] 10.222.0.98 () {56 vars in 1124 bytes} [Thu Aug 18 12:02:17 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:17.198932 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358739e-e1a5-4e2c-8c90-5c54b90f37d5 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:17.206182 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb851f-d806-41a4-8c56-6989c9cfe04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:17.206822 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb851f-d806-41a4-8c56-6989c9cfe04a None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:17.207110 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb851f-d806-41a4-8c56-6989c9cfe04a None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:17.207388 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb851f-d806-41a4-8c56-6989c9cfe04a None admin] PATH_INFO: `/v3/services/network` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:17.208054 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befb851f-d806-41a4-8c56-6989c9cfe04a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:17.209433 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befb851f-d806-41a4-8c56-6989c9cfe04a None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:17.218889 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-befb851f-d806-41a4-8c56-6989c9cfe04a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 12:02:17.218889 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:17.218889 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 12:02:17.218889 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 12:02:17.218889 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 12:02:17.218889 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 12:02:17.218889 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 12:02:17.218889 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:17.218889 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 12:02:17.218889 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:17.218889 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 12:02:17.218889 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:17.218889 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:17.218889 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:17.218889 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:17.218889 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:17.218889 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:17.220037 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:17.220037 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:17.220037 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:17.220037 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:17.220037 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:17.220037 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:17.220037 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 12:02:17.220037 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 12:02:17.220037 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 12:02:17.220037 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 12:02:17.220037 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:17.220037 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:17.220037 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:17.220037 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:17.220037 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:17.220037 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:17.221565 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:17.221565 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:17.221565 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:17.221565 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:17.221565 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:17.221565 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:17.221565 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:17.221565 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:17.221565 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:17.221565 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:17.221565 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 12:02:17.221565 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:17.221565 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 12:02:17.221565 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:17.221565 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 376/766] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:02:17 2022] GET /identity/v3/services/network => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:17.222459 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb851f-d806-41a4-8c56-6989c9cfe04a None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:17.277443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:17.294690 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:17.319283 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358739e-e1a5-4e2c-8c90-5c54b90f37d5 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:17.333475 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb851f-d806-41a4-8c56-6989c9cfe04a None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:17.357991 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a5f3d3-7ece-4cc1-a39e-4c9252d1abbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:17.358724 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5f3d3-7ece-4cc1-a39e-4c9252d1abbd None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:17.359012 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5f3d3-7ece-4cc1-a39e-4c9252d1abbd None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:17.359402 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5f3d3-7ece-4cc1-a39e-4c9252d1abbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:17.371264 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f97e5-5838-4e22-a98d-737beaa23305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:17.371909 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f97e5-5838-4e22-a98d-737beaa23305 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:17.372192 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f97e5-5838-4e22-a98d-737beaa23305 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:17.372780 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f97e5-5838-4e22-a98d-737beaa23305 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:17.373509 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6f97e5-5838-4e22-a98d-737beaa23305 None admin] RBAC: Adding query filter params (role.id=c53a0e13b58642f291c856efe1f0b87d, scope.project.id=6f676a5169464d83af4267db25025728, user.id=771591fe31cb4e8c9c4320c2a7b2719d) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:17.373821 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6f97e5-5838-4e22-a98d-737beaa23305 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:17.375236 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6f97e5-5838-4e22-a98d-737beaa23305 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:17.391828 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 377/767] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:02:17 2022] GET /identity/v3/role_assignments?user.id=771591fe31cb4e8c9c4320c2a7b2719d&scope.project.id=6f676a5169464d83af4267db25025728&role.id=c53a0e13b58642f291c856efe1f0b87d => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:17.391828 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f97e5-5838-4e22-a98d-737beaa23305 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:17.449212 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a5f3d3-7ece-4cc1-a39e-4c9252d1abbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:17.450794 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a5f3d3-7ece-4cc1-a39e-4c9252d1abbd None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:17.520223 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f97e5-5838-4e22-a98d-737beaa23305 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:17.539709 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:17.554913 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:17.563768 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d5cd0-affe-4457-9391-bd07ea40f459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:17.565882 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d5cd0-affe-4457-9391-bd07ea40f459 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:17.566325 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d5cd0-affe-4457-9391-bd07ea40f459 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:17.566958 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d5cd0-affe-4457-9391-bd07ea40f459 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:17.567713 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d5cd0-affe-4457-9391-bd07ea40f459 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:17.568085 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d5cd0-affe-4457-9391-bd07ea40f459 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:17.569486 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d5cd0-affe-4457-9391-bd07ea40f459 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:17.576797 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 378/768] 10.222.0.98 () {56 vars in 1124 bytes} [Thu Aug 18 12:02:17 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:17.579576 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 391/769] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:02:17 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 12:02:17.582113 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d5cd0-affe-4457-9391-bd07ea40f459 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:17.584617 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a5f3d3-7ece-4cc1-a39e-4c9252d1abbd None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:17.712981 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a5f3d3-7ece-4cc1-a39e-4c9252d1abbd None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:17.732744 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d5cd0-affe-4457-9391-bd07ea40f459 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:17.750302 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e295c334-a361-4a35-9629-2f79a950d03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:17.751194 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295c334-a361-4a35-9629-2f79a950d03d None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:17.752330 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295c334-a361-4a35-9629-2f79a950d03d None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:17.752620 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295c334-a361-4a35-9629-2f79a950d03d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:17.753323 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e295c334-a361-4a35-9629-2f79a950d03d None admin] RBAC: Adding query filter params (interface=public, service_id=4a8eb432fd614897a87689e9e4b8b74a, region_id=RegionOne) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:17.753634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e295c334-a361-4a35-9629-2f79a950d03d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:17.755446 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e295c334-a361-4a35-9629-2f79a950d03d None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:17.763995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 392/770] 10.222.0.98 () {56 vars in 1251 bytes} [Thu Aug 18 12:02:17 2022] GET /identity/v3/endpoints?service_id=4a8eb432fd614897a87689e9e4b8b74a&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:17.764827 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e295c334-a361-4a35-9629-2f79a950d03d None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:17.774365 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fbf7f9-7930-42b6-a31f-4e4aa6b56878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:17.774982 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fbf7f9-7930-42b6-a31f-4e4aa6b56878 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:17.775547 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fbf7f9-7930-42b6-a31f-4e4aa6b56878 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:17.775826 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fbf7f9-7930-42b6-a31f-4e4aa6b56878 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:17.776463 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fbf7f9-7930-42b6-a31f-4e4aa6b56878 None admin] RBAC: Adding query filter params (name=network) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:17.776771 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fbf7f9-7930-42b6-a31f-4e4aa6b56878 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:17.778130 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fbf7f9-7930-42b6-a31f-4e4aa6b56878 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:17.787202 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 379/771] 10.222.0.98 () {56 vars in 1112 bytes} [Thu Aug 18 12:02:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:17.787202 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9529d1-618a-40e6-b697-fe7a11eac062 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:17.787202 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9529d1-618a-40e6-b697-fe7a11eac062 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:17.787202 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9529d1-618a-40e6-b697-fe7a11eac062 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:17.787202 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 380/772] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:17.787202 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9529d1-618a-40e6-b697-fe7a11eac062 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:17.822453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:17.833794 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:17.882194 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e295c334-a361-4a35-9629-2f79a950d03d None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:17.898523 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9529d1-618a-40e6-b697-fe7a11eac062 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:17.917997 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89064674-564b-48e9-ac44-63f0820d045d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:17.918884 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89064674-564b-48e9-ac44-63f0820d045d None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:17.919206 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89064674-564b-48e9-ac44-63f0820d045d None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:17.919467 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89064674-564b-48e9-ac44-63f0820d045d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:17.931500 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89064674-564b-48e9-ac44-63f0820d045d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:17.933014 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89064674-564b-48e9-ac44-63f0820d045d None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:17.936239 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a209d95-50c9-4893-9096-7c0594909b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:17.936771 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a209d95-50c9-4893-9096-7c0594909b98 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:17.937052 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a209d95-50c9-4893-9096-7c0594909b98 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:17.937331 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a209d95-50c9-4893-9096-7c0594909b98 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:17.937945 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a209d95-50c9-4893-9096-7c0594909b98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:17.939788 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a209d95-50c9-4893-9096-7c0594909b98 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:17.951462 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 381/773] 10.222.0.98 () {56 vars in 1087 bytes} [Thu Aug 18 12:02:17 2022] GET /identity/v3/services => generated 772 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:17.951462 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a209d95-50c9-4893-9096-7c0594909b98 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:17.951750 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-89064674-564b-48e9-ac44-63f0820d045d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:17.951750 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:17.951750 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:17.951750 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:17.951750 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:17.951750 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:17.951750 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:17.951750 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:17.951750 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:17.951750 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:17.951750 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:17.951750 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:17.951750 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:17.951750 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:17.951750 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:17.951750 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:17.952648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:17.952648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:17.952648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:17.952648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:17.952648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:17.952648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:17.952648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:17.952648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:17.952648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:17.952648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:17.952648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:17.952648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:17.952648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:17.952648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:17.952648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:17.952648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:17.952648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:17.954151 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:17.954151 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:17.954151 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:17.954151 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 393/774] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:17.954151 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd784a3f-233b-4009-a182-eb2b88d2bba4 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:17.954151 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd784a3f-233b-4009-a182-eb2b88d2bba4 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:17.954151 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd784a3f-233b-4009-a182-eb2b88d2bba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:17.980954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-cd784a3f-233b-4009-a182-eb2b88d2bba4 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:17.982344 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-cd784a3f-233b-4009-a182-eb2b88d2bba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:18.084137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a209d95-50c9-4893-9096-7c0594909b98 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:18.092229 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:18.111887 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 394/775] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:02:18.113193 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd784a3f-233b-4009-a182-eb2b88d2bba4 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:18.124280 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70010dc-3739-41b0-bf87-808a29b655b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:18.125035 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70010dc-3739-41b0-bf87-808a29b655b0 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:18.125423 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70010dc-3739-41b0-bf87-808a29b655b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:18.125804 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70010dc-3739-41b0-bf87-808a29b655b0 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:18.126595 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70010dc-3739-41b0-bf87-808a29b655b0 None admin] RBAC: Adding query filter params (type=network) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:18.127083 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70010dc-3739-41b0-bf87-808a29b655b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:18.129270 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70010dc-3739-41b0-bf87-808a29b655b0 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:18.135375 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 382/776] 10.222.0.98 () {56 vars in 1112 bytes} [Thu Aug 18 12:02:17 2022] GET /identity/v3/services?type=network => generated 350 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:18.141077 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70010dc-3739-41b0-bf87-808a29b655b0 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:18.155745 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:18.272238 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd784a3f-233b-4009-a182-eb2b88d2bba4 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:18.286470 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70010dc-3739-41b0-bf87-808a29b655b0 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:18.343891 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70ea34-a119-4ee0-a7c0-89e0ec6aef37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:18.345404 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70ea34-a119-4ee0-a7c0-89e0ec6aef37 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:18.346692 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70ea34-a119-4ee0-a7c0-89e0ec6aef37 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:18.347720 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70ea34-a119-4ee0-a7c0-89e0ec6aef37 None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:18.348531 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de0f15-ae07-4d6f-90d2-41b9db15c43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:18.349736 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70ea34-a119-4ee0-a7c0-89e0ec6aef37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:18.350560 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de0f15-ae07-4d6f-90d2-41b9db15c43f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:18.351064 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70ea34-a119-4ee0-a7c0-89e0ec6aef37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:18.352658 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70ea34-a119-4ee0-a7c0-89e0ec6aef37 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:18.353157 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de0f15-ae07-4d6f-90d2-41b9db15c43f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:18.353642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de0f15-ae07-4d6f-90d2-41b9db15c43f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:18.354539 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de0f15-ae07-4d6f-90d2-41b9db15c43f None admin] RBAC: Adding query filter params (interface=public, service_id=b22de3744f524e4296f74a995b54df9d, region_id=RegionOne) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:18.356433 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de0f15-ae07-4d6f-90d2-41b9db15c43f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:18.357995 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de0f15-ae07-4d6f-90d2-41b9db15c43f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:18.370636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 395/777] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:18.374228 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 383/778] 10.222.0.98 () {56 vars in 1251 bytes} [Thu Aug 18 12:02:18 2022] GET /identity/v3/endpoints?service_id=b22de3744f524e4296f74a995b54df9d&interface=public®ion_id=RegionOne => generated 194 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:18.377112 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70ea34-a119-4ee0-a7c0-89e0ec6aef37 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:18.380287 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de0f15-ae07-4d6f-90d2-41b9db15c43f None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:18.418796 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:18.491298 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de0f15-ae07-4d6f-90d2-41b9db15c43f None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:18.501172 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:18.518872 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70ea34-a119-4ee0-a7c0-89e0ec6aef37 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:18.528827 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c6b11a-2384-49b3-8a8b-783999125506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:18.529590 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6b11a-2384-49b3-8a8b-783999125506 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:18.529949 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6b11a-2384-49b3-8a8b-783999125506 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:18.530299 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6b11a-2384-49b3-8a8b-783999125506 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:18.531000 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c6b11a-2384-49b3-8a8b-783999125506 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:18.532482 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c6b11a-2384-49b3-8a8b-783999125506 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:18.538578 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 384/779] 10.222.0.98 () {56 vars in 1087 bytes} [Thu Aug 18 12:02:18 2022] GET /identity/v3/services => generated 772 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:18.539527 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c6b11a-2384-49b3-8a8b-783999125506 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:18.562612 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf1b074-0b11-48b4-9c24-c691943acb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:18.564527 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf1b074-0b11-48b4-9c24-c691943acb3a None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:18.565051 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf1b074-0b11-48b4-9c24-c691943acb3a None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:18.565540 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf1b074-0b11-48b4-9c24-c691943acb3a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:18.574171 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf1b074-0b11-48b4-9c24-c691943acb3a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:18.576279 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf1b074-0b11-48b4-9c24-c691943acb3a None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:18.590513 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-bbf1b074-0b11-48b4-9c24-c691943acb3a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:18.590513 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:18.590513 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:18.590513 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:18.590513 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:18.590513 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:18.590513 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:18.590513 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:18.590513 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:18.590513 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:18.590513 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:18.590513 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:18.590513 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 12:02:18.590513 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 12:02:18.590513 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 12:02:18.591584 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 12:02:18.591584 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:18.591584 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:18.591584 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:18.591584 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:18.591584 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:18.591584 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:18.591584 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:18.591584 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:18.591584 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:18.591584 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:18.591584 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:18.591584 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:18.591584 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:18.591584 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:18.591584 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:18.591584 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:18.592472 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 12:02:18.592472 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 12:02:18.592472 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:02:18.592472 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:02:18.592472 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:02:18.592472 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:02:18.592472 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:18.592472 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:18.595067 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 396/780] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:02:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:18.601209 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf1b074-0b11-48b4-9c24-c691943acb3a None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:18.745425 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:18.753043 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c6b11a-2384-49b3-8a8b-783999125506 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:18.766941 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf1b074-0b11-48b4-9c24-c691943acb3a None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:18.799968 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecb369f-c738-4741-849e-fec4ce56b0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:18.800599 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb369f-c738-4741-849e-fec4ce56b0f2 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:18.800882 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb369f-c738-4741-849e-fec4ce56b0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:18.801162 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb369f-c738-4741-849e-fec4ce56b0f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:18.813173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9c296-a902-4e18-a51f-42366828bd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:18.814660 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9c296-a902-4e18-a51f-42366828bd9f None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:18.815513 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9c296-a902-4e18-a51f-42366828bd9f None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:18.815951 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecb369f-c738-4741-849e-fec4ce56b0f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:02:18.816218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9c296-a902-4e18-a51f-42366828bd9f None admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:18.817191 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd9c296-a902-4e18-a51f-42366828bd9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:18.817707 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd9c296-a902-4e18-a51f-42366828bd9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:18.819353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd9c296-a902-4e18-a51f-42366828bd9f None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:18.821415 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-7ecb369f-c738-4741-849e-fec4ce56b0f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:02:18.821415 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:18.821415 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:18.821415 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:18.821415 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:18.821415 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:18.821415 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:18.821415 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:18.821415 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:18.821415 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:18.821415 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:18.821415 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:18.821415 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:02:18.821415 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:02:18.821415 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:02:18.821415 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:02:18.822535 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:02:18.822535 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:02:18.822535 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:02:18.822535 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:02:18.822535 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:18.822535 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:18.822535 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:18.822535 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:18.822535 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:18.822535 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:18.822535 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:18.822535 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:18.822535 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:18.822535 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:18.822535 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:18.822535 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:18.823607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:18.823607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:18.823607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:18.823607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:18.823607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:02:18.823607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:02:18.823607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:02:18.823607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:02:18.823607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:02:18.823607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:02:18.823607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:02:18.823607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:18.823607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 385/781] 10.222.0.98 () {56 vars in 1096 bytes} [Thu Aug 18 12:02:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:18.828680 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecb369f-c738-4741-849e-fec4ce56b0f2 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:18.837624 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 397/782] 10.222.0.98 () {56 vars in 1148 bytes} [Thu Aug 18 12:02:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:02:18.843409 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9c296-a902-4e18-a51f-42366828bd9f None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:18.853639 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:18.951908 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecb369f-c738-4741-849e-fec4ce56b0f2 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:19.005415 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8175ef3a-2345-4c3f-8038-e46564ebe54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:19.007058 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175ef3a-2345-4c3f-8038-e46564ebe54c None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:19.008225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175ef3a-2345-4c3f-8038-e46564ebe54c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:19.008656 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175ef3a-2345-4c3f-8038-e46564ebe54c None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:19.009659 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8175ef3a-2345-4c3f-8038-e46564ebe54c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:19.011621 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8175ef3a-2345-4c3f-8038-e46564ebe54c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:19.013176 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8175ef3a-2345-4c3f-8038-e46564ebe54c None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:19.032141 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 386/783] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:02:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:02:19.038272 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8175ef3a-2345-4c3f-8038-e46564ebe54c None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:19.055042 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9c296-a902-4e18-a51f-42366828bd9f None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:19.079635 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:19.125667 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275544c-452b-4be1-8c10-d92f7ad1acbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:19.126362 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275544c-452b-4be1-8c10-d92f7ad1acbe None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:19.127270 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275544c-452b-4be1-8c10-d92f7ad1acbe None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:19.127689 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275544c-452b-4be1-8c10-d92f7ad1acbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:19.128534 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3275544c-452b-4be1-8c10-d92f7ad1acbe None admin] RBAC: Adding query filter params (role.id=563e3ed5c6164aae966b64cc302b114a, scope.project.id=6f676a5169464d83af4267db25025728, user.id=b6880278f7304051b9ef03e927637e74) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:19.128949 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3275544c-452b-4be1-8c10-d92f7ad1acbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:19.130525 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3275544c-452b-4be1-8c10-d92f7ad1acbe None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:19.144697 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 398/784] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:02:18 2022] GET /identity/v3/role_assignments?user.id=b6880278f7304051b9ef03e927637e74&scope.project.id=6f676a5169464d83af4267db25025728&role.id=563e3ed5c6164aae966b64cc302b114a => generated 259 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:02:19.174438 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8175ef3a-2345-4c3f-8038-e46564ebe54c None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:19.216300 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30da91a-f20e-4a07-85b6-46f83ec00dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:19.217294 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30da91a-f20e-4a07-85b6-46f83ec00dd3 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:19.217738 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30da91a-f20e-4a07-85b6-46f83ec00dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:19.218164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30da91a-f20e-4a07-85b6-46f83ec00dd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:19.230748 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30da91a-f20e-4a07-85b6-46f83ec00dd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:19.232629 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30da91a-f20e-4a07-85b6-46f83ec00dd3 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:19.249108 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-b30da91a-f20e-4a07-85b6-46f83ec00dd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:19.249108 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:19.249108 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:19.249108 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:19.249108 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:19.249108 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:19.249108 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:19.249108 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:19.249108 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:19.249108 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:19.249108 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:19.249108 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:19.249108 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:19.249108 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:19.249108 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:19.249108 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:19.250377 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:19.250377 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:19.250377 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:19.250377 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:19.250377 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:19.250377 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:19.250377 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:19.250377 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:19.250377 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:19.250377 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:19.250377 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:19.250377 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:19.250377 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:19.250377 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:19.250377 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:19.250377 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:19.250377 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:19.252117 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:19.252117 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:19.252117 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:19.254105 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 387/785] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:19.259509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275544c-452b-4be1-8c10-d92f7ad1acbe None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:19.390152 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:19.434050 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275544c-452b-4be1-8c10-d92f7ad1acbe None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:19.482760 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a2cda-59c0-485a-b803-b491db611a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:19.483381 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a2cda-59c0-485a-b803-b491db611a08 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:19.483695 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a2cda-59c0-485a-b803-b491db611a08 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:19.483974 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a2cda-59c0-485a-b803-b491db611a08 None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:19.484620 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a2cda-59c0-485a-b803-b491db611a08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:19.484927 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a2cda-59c0-485a-b803-b491db611a08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:19.486857 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a2cda-59c0-485a-b803-b491db611a08 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:19.502445 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 399/786] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:19.508127 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30da91a-f20e-4a07-85b6-46f83ec00dd3 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:19.629838 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:19.644604 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5fd6e-6325-4c01-9221-0cf910368a9f None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:19.645098 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5fd6e-6325-4c01-9221-0cf910368a9f None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:19.645098 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5fd6e-6325-4c01-9221-0cf910368a9f None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:19.646460 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 400/787] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:19.652748 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40334d-3bfd-435b-bcdf-43733fb6874d None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:19.654901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40334d-3bfd-435b-bcdf-43733fb6874d None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:19.654901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40334d-3bfd-435b-bcdf-43733fb6874d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:19.684987 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30da91a-f20e-4a07-85b6-46f83ec00dd3 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:19.694398 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-5f40334d-3bfd-435b-bcdf-43733fb6874d None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:19.695638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-5f40334d-3bfd-435b-bcdf-43733fb6874d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:19.724457 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab60ce1-c273-4766-9c91-46e9d594cef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:19.725459 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab60ce1-c273-4766-9c91-46e9d594cef9 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:19.725949 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab60ce1-c273-4766-9c91-46e9d594cef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:19.726450 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab60ce1-c273-4766-9c91-46e9d594cef9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:19.735466 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab60ce1-c273-4766-9c91-46e9d594cef9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:19.738122 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab60ce1-c273-4766-9c91-46e9d594cef9 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:19.752444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-4ab60ce1-c273-4766-9c91-46e9d594cef9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 12:02:19.752444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:19.752444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:19.752444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:19.752444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:19.752444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:19.752444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:19.752444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:19.752444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:19.752444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:19.752444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:19.752444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:19.752444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 12:02:19.752444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 12:02:19.752444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 12:02:19.752444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 12:02:19.753454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:19.753454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:19.753454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:02:19.753454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:19.753454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:02:19.753454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:19.753454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:19.753454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:19.753454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:19.753454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:19.753454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:19.753454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:19.753454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:19.753454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:19.753454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:19.753454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:19.753454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:19.753454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:19.754418 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:19.754418 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:19.754418 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 12:02:19.754418 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 12:02:19.754418 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 12:02:19.754418 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 12:02:19.754418 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 12:02:19.754418 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 12:02:19.754418 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 12:02:19.754418 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:19.756984 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 388/788] 10.222.0.98 () {56 vars in 1093 bytes} [Thu Aug 18 12:02:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:19.763679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab60ce1-c273-4766-9c91-46e9d594cef9 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:19.845096 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 401/789] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 12:02:19.923573 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab60ce1-c273-4766-9c91-46e9d594cef9 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:19.957080 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:19.964643 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc2af07-9762-4f08-8e65-4ae1c165196e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:19.969443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2af07-9762-4f08-8e65-4ae1c165196e None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:19.969443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2af07-9762-4f08-8e65-4ae1c165196e None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:19.969443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2af07-9762-4f08-8e65-4ae1c165196e None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:19.969443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc2af07-9762-4f08-8e65-4ae1c165196e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:19.969443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc2af07-9762-4f08-8e65-4ae1c165196e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:19.969443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc2af07-9762-4f08-8e65-4ae1c165196e None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:20.017118 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 389/790] 10.222.0.98 () {56 vars in 1133 bytes} [Thu Aug 18 12:02:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:02:20.022639 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f40334d-3bfd-435b-bcdf-43733fb6874d None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:20.090800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:20.136100 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f40334d-3bfd-435b-bcdf-43733fb6874d None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:20.171012 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9478d4-e115-42a5-974a-05e6741bd082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:20.171653 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9478d4-e115-42a5-974a-05e6741bd082 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:20.172203 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9478d4-e115-42a5-974a-05e6741bd082 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:20.172477 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9478d4-e115-42a5-974a-05e6741bd082 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:20.243073 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9478d4-e115-42a5-974a-05e6741bd082 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:20.244588 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9478d4-e115-42a5-974a-05e6741bd082 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:20.286758 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc2af07-9762-4f08-8e65-4ae1c165196e None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:20.307842 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:20.402638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 402/791] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:02:20 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 12:02:20.409017 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9478d4-e115-42a5-974a-05e6741bd082 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:20.414515 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc2af07-9762-4f08-8e65-4ae1c165196e None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:20.457617 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb431fd-8ccb-4e95-9cd4-4ca3ebc5891e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:20.458662 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb431fd-8ccb-4e95-9cd4-4ca3ebc5891e None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:20.459154 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb431fd-8ccb-4e95-9cd4-4ca3ebc5891e None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:20.460141 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb431fd-8ccb-4e95-9cd4-4ca3ebc5891e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:20.473313 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb431fd-8ccb-4e95-9cd4-4ca3ebc5891e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:02:20.479311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-3bb431fd-8ccb-4e95-9cd4-4ca3ebc5891e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:02:20.479311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:20.479311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:20.479311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:20.479311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:20.479311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:20.479311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:20.479311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:20.479311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:20.479311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:20.479311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:20.479311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:20.479311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:02:20.479311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:02:20.479311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:02:20.479311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:02:20.480283 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:02:20.480283 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:02:20.480283 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:02:20.480283 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:02:20.480283 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:20.480283 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:20.480283 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:20.480283 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:20.480283 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:20.480283 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:20.480283 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:20.480283 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:20.480283 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:20.480283 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:20.480283 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:20.480283 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:20.481135 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:20.481135 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:20.481135 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:20.481135 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:20.481135 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:02:20.481135 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:02:20.481135 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:02:20.481135 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:02:20.481135 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:02:20.481135 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:02:20.481135 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:02:20.481135 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:20.484555 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 390/792] 10.222.0.98 () {56 vars in 1096 bytes} [Thu Aug 18 12:02:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:02:20.493980 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb431fd-8ccb-4e95-9cd4-4ca3ebc5891e None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:20.572766 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:20.573752 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9478d4-e115-42a5-974a-05e6741bd082 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:20.613005 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77eb58-457f-48d7-ae7a-257171822319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:20.613766 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77eb58-457f-48d7-ae7a-257171822319 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:20.614047 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77eb58-457f-48d7-ae7a-257171822319 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:20.614322 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77eb58-457f-48d7-ae7a-257171822319 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:20.625897 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f77eb58-457f-48d7-ae7a-257171822319 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:20.628056 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f77eb58-457f-48d7-ae7a-257171822319 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:20.640919 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb431fd-8ccb-4e95-9cd4-4ca3ebc5891e None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:20.645712 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-3f77eb58-457f-48d7-ae7a-257171822319 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:20.645712 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:20.645712 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:20.645712 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:20.645712 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:20.645712 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:20.645712 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:20.645712 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:20.645712 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:20.645712 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:20.645712 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:20.645712 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:20.645712 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:20.645712 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:20.645712 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:20.645712 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:20.646995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:20.646995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:20.646995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:20.646995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:20.646995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:20.646995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:20.646995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:20.646995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:20.646995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:20.646995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:20.646995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:20.646995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:20.646995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:20.646995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:20.646995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:20.646995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:20.646995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:20.648368 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:20.648368 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:20.648368 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:20.648368 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 403/793] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:20.648600 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07c6b7-5f9f-452c-8ded-7217072b35d8 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:20.648801 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07c6b7-5f9f-452c-8ded-7217072b35d8 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:20.650577 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07c6b7-5f9f-452c-8ded-7217072b35d8 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:20.654917 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 404/794] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:20.655797 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07c6b7-5f9f-452c-8ded-7217072b35d8 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:20.679536 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:20.683064 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9298148-b238-47b7-aab0-1fc87d765474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:20.684030 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9298148-b238-47b7-aab0-1fc87d765474 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:20.684783 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9298148-b238-47b7-aab0-1fc87d765474 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:20.685209 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9298148-b238-47b7-aab0-1fc87d765474 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:20.686026 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9298148-b238-47b7-aab0-1fc87d765474 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:20.686492 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9298148-b238-47b7-aab0-1fc87d765474 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:20.688477 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9298148-b238-47b7-aab0-1fc87d765474 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:20.698900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 391/795] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:02:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:20.700451 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baffbd16-c3ee-4ab1-8a36-d9f8331a8076 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:20.700743 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baffbd16-c3ee-4ab1-8a36-d9f8331a8076 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:20.701022 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baffbd16-c3ee-4ab1-8a36-d9f8331a8076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:20.722680 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-baffbd16-c3ee-4ab1-8a36-d9f8331a8076 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:20.724465 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-baffbd16-c3ee-4ab1-8a36-d9f8331a8076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:20.807706 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07c6b7-5f9f-452c-8ded-7217072b35d8 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:20.808182 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 392/796] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:02:20.809481 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baffbd16-c3ee-4ab1-8a36-d9f8331a8076 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:20.856679 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0dd20-f90a-491d-afd0-4ef1beb54891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:20.857276 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0dd20-f90a-491d-afd0-4ef1beb54891 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:20.857556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0dd20-f90a-491d-afd0-4ef1beb54891 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:20.857829 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0dd20-f90a-491d-afd0-4ef1beb54891 None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:20.859445 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0dd20-f90a-491d-afd0-4ef1beb54891 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:20.859445 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0dd20-f90a-491d-afd0-4ef1beb54891 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:20.862080 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0dd20-f90a-491d-afd0-4ef1beb54891 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:20.875319 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 405/797] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:20.880297 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0dd20-f90a-491d-afd0-4ef1beb54891 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:20.959810 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:20.993566 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0dd20-f90a-491d-afd0-4ef1beb54891 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:21.032322 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327a52e-1332-4a71-a2b3-c5654f1282dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:21.033622 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327a52e-1332-4a71-a2b3-c5654f1282dd None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:21.033937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327a52e-1332-4a71-a2b3-c5654f1282dd None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:21.034219 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327a52e-1332-4a71-a2b3-c5654f1282dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:21.041798 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3327a52e-1332-4a71-a2b3-c5654f1282dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:21.043272 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3327a52e-1332-4a71-a2b3-c5654f1282dd None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:21.055937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-3327a52e-1332-4a71-a2b3-c5654f1282dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:21.055937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:21.055937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:21.055937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:21.055937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:21.055937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:21.055937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:21.055937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:21.055937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:21.055937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:21.055937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:21.055937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:21.055937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 12:02:21.055937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 12:02:21.055937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 12:02:21.056893 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 12:02:21.056893 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:21.056893 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:21.056893 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:21.056893 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:21.056893 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:21.056893 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:21.056893 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:21.056893 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:21.056893 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:21.056893 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:21.056893 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:21.056893 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:21.056893 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:21.056893 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:21.056893 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:21.056893 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:21.057845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 12:02:21.057845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 12:02:21.057845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:02:21.057845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:02:21.057845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:02:21.057845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:02:21.057845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:21.057845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:21.057845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 406/798] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:02:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:21.059096 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0eca42-2103-4fae-8990-3dbffb9b81bd None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:21.059096 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0eca42-2103-4fae-8990-3dbffb9b81bd None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:21.063437 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0eca42-2103-4fae-8990-3dbffb9b81bd None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:21.063437 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 407/799] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:21.064960 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0eca42-2103-4fae-8990-3dbffb9b81bd None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:21.069041 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:21.142457 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baffbd16-c3ee-4ab1-8a36-d9f8331a8076 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:21.192427 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba57b5-f930-4f89-8cf4-f52bb138fa6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:21.193070 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba57b5-f930-4f89-8cf4-f52bb138fa6a None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:21.193353 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba57b5-f930-4f89-8cf4-f52bb138fa6a None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:21.193629 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba57b5-f930-4f89-8cf4-f52bb138fa6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:21.204356 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bba57b5-f930-4f89-8cf4-f52bb138fa6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:21.205890 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bba57b5-f930-4f89-8cf4-f52bb138fa6a None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:21.212449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0eca42-2103-4fae-8990-3dbffb9b81bd None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:21.221546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-9bba57b5-f930-4f89-8cf4-f52bb138fa6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:21.221546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:21.221546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:21.221546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:21.221546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:21.221546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:21.221546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:21.221546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:21.221546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:21.221546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:21.221546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:21.221546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:21.221546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:21.221546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:21.221546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:21.221546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:21.222576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:21.222576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:21.222576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:21.222576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:21.222576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:21.222576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:21.222576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:21.222576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:21.222576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:21.222576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:21.222576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:21.222576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:21.222576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:21.222576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:21.222576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:21.222576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:21.222576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:21.224481 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:21.224481 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:21.224481 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:21.224481 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 393/800] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 414 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:21.224481 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac74814-8155-4d37-89f2-2c3ab950f625 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:21.224768 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac74814-8155-4d37-89f2-2c3ab950f625 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:21.225428 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac74814-8155-4d37-89f2-2c3ab950f625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:21.244510 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-8ac74814-8155-4d37-89f2-2c3ab950f625 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:21.245735 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-8ac74814-8155-4d37-89f2-2c3ab950f625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:21.250437 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:21.256553 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaad49d-4061-49f8-8eb0-26afb61f0d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:21.257174 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaad49d-4061-49f8-8eb0-26afb61f0d78 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:21.257434 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaad49d-4061-49f8-8eb0-26afb61f0d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:21.257746 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaad49d-4061-49f8-8eb0-26afb61f0d78 None admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:21.258392 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaad49d-4061-49f8-8eb0-26afb61f0d78 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:21.258697 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaad49d-4061-49f8-8eb0-26afb61f0d78 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:21.260175 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaad49d-4061-49f8-8eb0-26afb61f0d78 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:21.272466 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 408/801] 10.222.0.98 () {56 vars in 1148 bytes} [Thu Aug 18 12:02:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:21.273453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaad49d-4061-49f8-8eb0-26afb61f0d78 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:21.327060 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 394/802] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:02:21.328183 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac74814-8155-4d37-89f2-2c3ab950f625 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:21.364305 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:21.390464 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaad49d-4061-49f8-8eb0-26afb61f0d78 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:21.424422 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be82c86-e3d1-41fc-81d1-44af03989c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:21.425048 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be82c86-e3d1-41fc-81d1-44af03989c98 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:21.425334 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be82c86-e3d1-41fc-81d1-44af03989c98 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:21.425614 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be82c86-e3d1-41fc-81d1-44af03989c98 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:21.426193 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be82c86-e3d1-41fc-81d1-44af03989c98 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:21.427598 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be82c86-e3d1-41fc-81d1-44af03989c98 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:21.438770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-0be82c86-e3d1-41fc-81d1-44af03989c98 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 12:02:21.438770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:21.438770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 12:02:21.438770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 12:02:21.438770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 12:02:21.438770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 12:02:21.438770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 12:02:21.438770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:21.438770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 12:02:21.438770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:21.438770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 12:02:21.438770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:21.438770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:21.438770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:21.438770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:21.438770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:21.438770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:21.440353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:21.440353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:21.440353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:21.440353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:21.440353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:21.440353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:21.440353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 12:02:21.440353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 12:02:21.440353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 12:02:21.440353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 12:02:21.440353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:21.440353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:21.440353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:21.440353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:21.440353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:21.440353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:21.441218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:21.441218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:21.441218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:21.441218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:21.441218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:21.441218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:21.441218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:21.441218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:21.441218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:21.441218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:21.441218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 12:02:21.441218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:21.441218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 18 12:02:21.441218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:21.441218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 409/803] 10.222.0.98 () {56 vars in 1129 bytes} [Thu Aug 18 12:02:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:21.442460 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be82c86-e3d1-41fc-81d1-44af03989c98 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:21.450155 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac74814-8155-4d37-89f2-2c3ab950f625 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:21.490769 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d43610b-8aee-45a2-ba53-cba5a1bb1b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:21.491715 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43610b-8aee-45a2-ba53-cba5a1bb1b49 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:21.492176 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43610b-8aee-45a2-ba53-cba5a1bb1b49 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:21.492718 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43610b-8aee-45a2-ba53-cba5a1bb1b49 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:21.493656 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d43610b-8aee-45a2-ba53-cba5a1bb1b49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:21.493964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d43610b-8aee-45a2-ba53-cba5a1bb1b49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:21.495316 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d43610b-8aee-45a2-ba53-cba5a1bb1b49 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:21.503923 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:21.508379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 395/804] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:21.509728 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6dd24-c6d9-4d3e-bed2-84715a69e128 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:21.510020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6dd24-c6d9-4d3e-bed2-84715a69e128 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:21.515599 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6dd24-c6d9-4d3e-bed2-84715a69e128 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:21.515599 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 396/805] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:21.515599 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd6dd24-c6d9-4d3e-bed2-84715a69e128 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:21.558900 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be82c86-e3d1-41fc-81d1-44af03989c98 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:21.596141 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55d729-1174-48fd-9b20-a219c99b0f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:21.596659 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55d729-1174-48fd-9b20-a219c99b0f88 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:21.596942 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55d729-1174-48fd-9b20-a219c99b0f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:21.597218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55d729-1174-48fd-9b20-a219c99b0f88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:21.597944 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b55d729-1174-48fd-9b20-a219c99b0f88 None admin] RBAC: Adding query filter params (role.id=c53a0e13b58642f291c856efe1f0b87d, scope.project.id=6f676a5169464d83af4267db25025728, user.id=171c72a6b29e418bbb4ea5c6faffe3b3) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:21.598599 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b55d729-1174-48fd-9b20-a219c99b0f88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:21.599737 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b55d729-1174-48fd-9b20-a219c99b0f88 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:21.606056 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 410/806] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:02:21 2022] GET /identity/v3/role_assignments?user.id=171c72a6b29e418bbb4ea5c6faffe3b3&scope.project.id=6f676a5169464d83af4267db25025728&role.id=c53a0e13b58642f291c856efe1f0b87d => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:21.606888 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55d729-1174-48fd-9b20-a219c99b0f88 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:21.609008 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:21.628672 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd6dd24-c6d9-4d3e-bed2-84715a69e128 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:21.670036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a641794e-471f-4fae-9aac-c8ffacf0d6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:21.670688 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641794e-471f-4fae-9aac-c8ffacf0d6de None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:21.670976 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641794e-471f-4fae-9aac-c8ffacf0d6de None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:21.671333 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641794e-471f-4fae-9aac-c8ffacf0d6de None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:21.672367 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a641794e-471f-4fae-9aac-c8ffacf0d6de None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:21.672681 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a641794e-471f-4fae-9aac-c8ffacf0d6de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:21.674885 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a641794e-471f-4fae-9aac-c8ffacf0d6de None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:21.680767 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 397/807] 10.222.0.98 () {56 vars in 1124 bytes} [Thu Aug 18 12:02:21 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:21.682053 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79a07cd-9e3f-4c9d-bd99-702f4c6d011e None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:21.682343 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79a07cd-9e3f-4c9d-bd99-702f4c6d011e None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:21.682621 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79a07cd-9e3f-4c9d-bd99-702f4c6d011e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:21.705874 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-a79a07cd-9e3f-4c9d-bd99-702f4c6d011e None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:21.707115 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-a79a07cd-9e3f-4c9d-bd99-702f4c6d011e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:21.734493 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55d729-1174-48fd-9b20-a219c99b0f88 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:21.757904 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:21.774892 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c31917-cdba-44f4-bfa9-58a8f5cfaf47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:21.775858 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c31917-cdba-44f4-bfa9-58a8f5cfaf47 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:21.776147 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c31917-cdba-44f4-bfa9-58a8f5cfaf47 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:21.776425 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c31917-cdba-44f4-bfa9-58a8f5cfaf47 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:21.782667 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 398/808] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:02:21.784213 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79a07cd-9e3f-4c9d-bd99-702f4c6d011e None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:21.785178 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c31917-cdba-44f4-bfa9-58a8f5cfaf47 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:21.786647 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c31917-cdba-44f4-bfa9-58a8f5cfaf47 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:21.798791 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-e4c31917-cdba-44f4-bfa9-58a8f5cfaf47 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 18 12:02:21.798791 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:21.798791 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:21.798791 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:21.798791 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:21.798791 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:21.798791 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:21.798791 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:21.798791 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:21.798791 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:21.798791 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:21.798791 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:21.798791 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 12:02:21.798791 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 12:02:21.798791 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 12:02:21.798791 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 12:02:21.799827 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:21.799827 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:21.799827 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:02:21.799827 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:21.799827 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:02:21.799827 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:21.799827 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:21.799827 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:21.799827 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:21.799827 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:21.799827 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:21.799827 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:21.799827 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:21.799827 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:21.799827 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:21.799827 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:21.799827 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:21.799827 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:21.800845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:21.800845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:21.800845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 12:02:21.800845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 12:02:21.800845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 12:02:21.800845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 12:02:21.800845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 12:02:21.800845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 12:02:21.800845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 18 12:02:21.800845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:21.804637 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 411/809] 10.222.0.98 () {56 vars in 1096 bytes} [Thu Aug 18 12:02:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:21.805503 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c31917-cdba-44f4-bfa9-58a8f5cfaf47 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:21.880392 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:21.902255 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79a07cd-9e3f-4c9d-bd99-702f4c6d011e None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:21.938125 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3fb816-4f44-46bf-9dd4-11af094cf1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:21.938934 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fb816-4f44-46bf-9dd4-11af094cf1a8 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:21.939342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fb816-4f44-46bf-9dd4-11af094cf1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:21.939602 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c31917-cdba-44f4-bfa9-58a8f5cfaf47 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:21.939721 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fb816-4f44-46bf-9dd4-11af094cf1a8 None admin] PATH_INFO: `/v3/services/network` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:21.940244 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3fb816-4f44-46bf-9dd4-11af094cf1a8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:21.941667 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3fb816-4f44-46bf-9dd4-11af094cf1a8 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:21.958116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-6c3fb816-4f44-46bf-9dd4-11af094cf1a8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 12:02:21.958116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:21.958116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 12:02:21.958116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 12:02:21.958116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 12:02:21.958116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 12:02:21.958116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 12:02:21.958116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:21.958116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 12:02:21.958116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:21.958116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 12:02:21.958116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:21.958116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:21.958116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:21.958116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:21.958116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:21.958116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:21.959373 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:21.959373 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:21.959373 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:21.959373 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:21.959373 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:21.959373 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:21.959373 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 12:02:21.959373 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 12:02:21.959373 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 12:02:21.959373 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 12:02:21.959373 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:21.959373 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:21.959373 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:21.959373 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:21.959373 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:21.959373 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:21.960444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:21.960444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:21.960444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:21.960444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:21.960444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:21.960444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:21.960444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:21.960444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:21.960444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:21.960444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:21.960444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 12:02:21.960444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:21.960444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 18 12:02:21.960444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:21.963314 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 399/810] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:02:21 2022] GET /identity/v3/services/network => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:21.964368 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3fb816-4f44-46bf-9dd4-11af094cf1a8 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:21.984400 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c7bba-b6ea-4a9c-b221-05040e63ecdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:21.985532 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c7bba-b6ea-4a9c-b221-05040e63ecdf None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:21.986019 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c7bba-b6ea-4a9c-b221-05040e63ecdf None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:21.986442 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c7bba-b6ea-4a9c-b221-05040e63ecdf None admin] PATH_INFO: `/v3/services` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:21.987356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8c7bba-b6ea-4a9c-b221-05040e63ecdf None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:21.988087 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8c7bba-b6ea-4a9c-b221-05040e63ecdf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:21.989599 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8c7bba-b6ea-4a9c-b221-05040e63ecdf None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:21.996769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 412/811] 10.222.0.98 () {56 vars in 1124 bytes} [Thu Aug 18 12:02:21 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:21.997808 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c7bba-b6ea-4a9c-b221-05040e63ecdf None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:22.056863 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:22.084151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3fb816-4f44-46bf-9dd4-11af094cf1a8 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:22.113993 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c7bba-b6ea-4a9c-b221-05040e63ecdf None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:22.121764 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2aba0-05cb-4ded-afde-8c0d48d7ff7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:22.122372 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2aba0-05cb-4ded-afde-8c0d48d7ff7c None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:22.122649 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2aba0-05cb-4ded-afde-8c0d48d7ff7c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:22.122917 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2aba0-05cb-4ded-afde-8c0d48d7ff7c None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:22.123684 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2aba0-05cb-4ded-afde-8c0d48d7ff7c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:22.123989 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2aba0-05cb-4ded-afde-8c0d48d7ff7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:22.125403 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2aba0-05cb-4ded-afde-8c0d48d7ff7c None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:22.141321 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:22.145573 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 400/812] 10.222.0.98 () {56 vars in 1135 bytes} [Thu Aug 18 12:02:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:22.151168 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2aba0-05cb-4ded-afde-8c0d48d7ff7c None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:22.151723 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db2fe95-4a20-41ef-b4fa-441100787e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:22.156950 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2fe95-4a20-41ef-b4fa-441100787e8c None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:22.156950 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2fe95-4a20-41ef-b4fa-441100787e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:22.156950 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2fe95-4a20-41ef-b4fa-441100787e8c None admin] PATH_INFO: `/v3/services` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:22.156950 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db2fe95-4a20-41ef-b4fa-441100787e8c None admin] RBAC: Adding query filter params (name=network) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:22.156950 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db2fe95-4a20-41ef-b4fa-441100787e8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:22.156950 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db2fe95-4a20-41ef-b4fa-441100787e8c None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:22.165158 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 413/813] 10.222.0.98 () {56 vars in 1112 bytes} [Thu Aug 18 12:02:21 2022] GET /identity/v3/services?name=network => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:22.165158 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db2fe95-4a20-41ef-b4fa-441100787e8c None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:22.266765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2aba0-05cb-4ded-afde-8c0d48d7ff7c None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:22.274143 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db2fe95-4a20-41ef-b4fa-441100787e8c None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:22.284227 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:22.307550 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff34245-e1cb-4ea6-b559-35622789c2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:22.308156 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff34245-e1cb-4ea6-b559-35622789c2de None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:22.308439 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff34245-e1cb-4ea6-b559-35622789c2de None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:22.308717 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff34245-e1cb-4ea6-b559-35622789c2de None admin] PATH_INFO: `/v3/endpoints` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:22.309384 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff34245-e1cb-4ea6-b559-35622789c2de None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:22.310146 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947132b-e0e1-423c-8944-4adff751972e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:22.310731 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff34245-e1cb-4ea6-b559-35622789c2de None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:22.310835 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947132b-e0e1-423c-8944-4adff751972e None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:22.311046 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947132b-e0e1-423c-8944-4adff751972e None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:22.311318 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947132b-e0e1-423c-8944-4adff751972e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:22.322523 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a947132b-e0e1-423c-8944-4adff751972e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:02:22.327681 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-a947132b-e0e1-423c-8944-4adff751972e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:02:22.327681 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:22.327681 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:22.327681 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:22.327681 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:22.327681 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:22.327681 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:22.327681 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:22.327681 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:22.327681 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:22.327681 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:22.327681 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:22.327681 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:02:22.327681 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:02:22.327681 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:02:22.327681 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:02:22.335169 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:02:22.335169 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:02:22.335169 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:02:22.335169 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:02:22.335169 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:22.335169 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:22.335169 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:22.335169 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:22.335169 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:22.335169 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:22.335169 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:22.335169 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:22.335169 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:22.335169 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:22.335169 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:22.335169 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:22.336078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:22.336078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:22.336078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:22.336078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:22.336078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:02:22.336078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:02:22.336078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:02:22.336078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:02:22.336078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:02:22.336078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:02:22.336078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:02:22.336078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:22.336078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 414/814] 10.222.0.98 () {56 vars in 1102 bytes} [Thu Aug 18 12:02:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:22.336078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947132b-e0e1-423c-8944-4adff751972e None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:22.425054 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 401/815] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:02:22 2022] POST /identity/v3/endpoints => generated 346 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:02:22.434378 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947132b-e0e1-423c-8944-4adff751972e None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:22.436564 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff34245-e1cb-4ea6-b559-35622789c2de None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:22.492263 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1736ba5f-0627-42cf-b24c-cb50d3bdd488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:22.494134 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736ba5f-0627-42cf-b24c-cb50d3bdd488 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:22.494695 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736ba5f-0627-42cf-b24c-cb50d3bdd488 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:22.495219 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736ba5f-0627-42cf-b24c-cb50d3bdd488 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:22.513940 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1736ba5f-0627-42cf-b24c-cb50d3bdd488 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:22.514457 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:22.516966 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1736ba5f-0627-42cf-b24c-cb50d3bdd488 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:22.542573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-1736ba5f-0627-42cf-b24c-cb50d3bdd488 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:22.542573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:22.542573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:22.542573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:22.542573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:22.542573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:22.542573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:22.542573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:22.542573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:22.542573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:22.542573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:22.542573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:22.542573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:22.542573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:22.542573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:22.542573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:22.543689 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:22.543689 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:22.543689 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:22.543689 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:22.543689 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:22.543689 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:22.543689 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:22.543689 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:22.543689 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:22.543689 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:22.543689 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:22.543689 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:22.543689 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:22.543689 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:22.543689 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:22.543689 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:22.543689 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:22.544595 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:22.544595 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:22.544595 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:22.547618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 415/816] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:22.549671 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1736ba5f-0627-42cf-b24c-cb50d3bdd488 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:22.568697 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff34245-e1cb-4ea6-b559-35622789c2de None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:22.576563 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:22.615670 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d59a5d-a7c7-4837-880b-c45ec7235136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:22.616298 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d59a5d-a7c7-4837-880b-c45ec7235136 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:22.616566 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d59a5d-a7c7-4837-880b-c45ec7235136 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:22.617028 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d59a5d-a7c7-4837-880b-c45ec7235136 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:22.617488 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d59a5d-a7c7-4837-880b-c45ec7235136 None admin] RBAC: Adding query filter params (type=network) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:22.617795 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d59a5d-a7c7-4837-880b-c45ec7235136 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:22.619234 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d59a5d-a7c7-4837-880b-c45ec7235136 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:22.625642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 402/817] 10.222.0.98 () {56 vars in 1112 bytes} [Thu Aug 18 12:02:22 2022] GET /identity/v3/services?type=network => generated 350 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:22.626452 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d59a5d-a7c7-4837-880b-c45ec7235136 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:22.700912 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1736ba5f-0627-42cf-b24c-cb50d3bdd488 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:22.738588 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc2d22-0fcc-4e3d-bbd9-437f88e7b222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:22.739254 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc2d22-0fcc-4e3d-bbd9-437f88e7b222 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:22.739530 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc2d22-0fcc-4e3d-bbd9-437f88e7b222 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:22.739780 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc2d22-0fcc-4e3d-bbd9-437f88e7b222 None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:22.740564 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cc2d22-0fcc-4e3d-bbd9-437f88e7b222 None admin] RBAC: Adding query filter params (name=service) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:22.740877 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cc2d22-0fcc-4e3d-bbd9-437f88e7b222 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:22.742500 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cc2d22-0fcc-4e3d-bbd9-437f88e7b222 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:22.756803 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 416/818] 10.222.0.98 () {56 vars in 1103 bytes} [Thu Aug 18 12:02:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:22.756803 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc2d22-0fcc-4e3d-bbd9-437f88e7b222 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:22.766850 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d59a5d-a7c7-4837-880b-c45ec7235136 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:22.797405 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:22.819382 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87034c7-49b4-4484-a43c-0e6da9d54bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:22.821056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87034c7-49b4-4484-a43c-0e6da9d54bef None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:22.821718 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87034c7-49b4-4484-a43c-0e6da9d54bef None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:22.822312 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87034c7-49b4-4484-a43c-0e6da9d54bef None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:22.823192 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87034c7-49b4-4484-a43c-0e6da9d54bef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:22.823841 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87034c7-49b4-4484-a43c-0e6da9d54bef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:22.825261 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87034c7-49b4-4484-a43c-0e6da9d54bef None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:22.843883 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 403/819] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:22.844537 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87034c7-49b4-4484-a43c-0e6da9d54bef None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:22.857007 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:22.880212 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc2d22-0fcc-4e3d-bbd9-437f88e7b222 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:22.917935 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d529fb3-c21d-4b92-b3e8-160afec8fdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:22.918554 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d529fb3-c21d-4b92-b3e8-160afec8fdb3 None admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:22.918843 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d529fb3-c21d-4b92-b3e8-160afec8fdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:22.919191 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d529fb3-c21d-4b92-b3e8-160afec8fdb3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:22.920089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d529fb3-c21d-4b92-b3e8-160afec8fdb3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:22.921646 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d529fb3-c21d-4b92-b3e8-160afec8fdb3 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:22.978320 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87034c7-49b4-4484-a43c-0e6da9d54bef None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:23.018650 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 417/820] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:02:22 2022] POST /identity/v3/endpoints => generated 337 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:02:23.018650 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d529fb3-c21d-4b92-b3e8-160afec8fdb3 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:23.028649 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5fbc91-2c95-4fed-8987-42a324c445a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:23.029314 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5fbc91-2c95-4fed-8987-42a324c445a0 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:23.029640 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5fbc91-2c95-4fed-8987-42a324c445a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:23.029920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5fbc91-2c95-4fed-8987-42a324c445a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:23.052389 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5fbc91-2c95-4fed-8987-42a324c445a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:23.055146 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5fbc91-2c95-4fed-8987-42a324c445a0 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:23.083572 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-ee5fbc91-2c95-4fed-8987-42a324c445a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:23.083572 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:23.083572 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:23.083572 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:23.083572 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:23.083572 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:23.083572 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:23.083572 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:23.083572 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:23.083572 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:23.083572 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:23.083572 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:23.083572 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:23.083572 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:23.083572 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:23.083572 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:23.084681 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:23.084681 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:23.084681 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:23.084681 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:23.084681 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:23.084681 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:23.084681 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:23.084681 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:23.084681 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:23.084681 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:23.084681 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:23.084681 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:23.084681 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:23.084681 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:23.084681 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:23.084681 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:23.084681 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:23.085628 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:23.085628 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:23.085628 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:23.085628 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 404/821] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:23.093153 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5fbc91-2c95-4fed-8987-42a324c445a0 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:23.155397 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:23.256973 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d529fb3-c21d-4b92-b3e8-160afec8fdb3 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:23.309031 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5fbc91-2c95-4fed-8987-42a324c445a0 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:23.324370 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae397b9-e77d-4c10-a46f-3c24c9707af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:23.326134 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae397b9-e77d-4c10-a46f-3c24c9707af9 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:23.326965 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae397b9-e77d-4c10-a46f-3c24c9707af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:23.328361 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae397b9-e77d-4c10-a46f-3c24c9707af9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:23.333732 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:23.347174 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae397b9-e77d-4c10-a46f-3c24c9707af9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:23.351094 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae397b9-e77d-4c10-a46f-3c24c9707af9 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:23.359780 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd829580-07ec-4ee3-8eac-7b559d83ba46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:23.360393 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd829580-07ec-4ee3-8eac-7b559d83ba46 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:23.360660 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd829580-07ec-4ee3-8eac-7b559d83ba46 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:23.360936 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd829580-07ec-4ee3-8eac-7b559d83ba46 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:23.361611 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd829580-07ec-4ee3-8eac-7b559d83ba46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:23.361912 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd829580-07ec-4ee3-8eac-7b559d83ba46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:23.363271 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd829580-07ec-4ee3-8eac-7b559d83ba46 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:23.381049 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-aae397b9-e77d-4c10-a46f-3c24c9707af9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:23.381049 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:23.381049 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:23.381049 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:23.381049 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:23.381049 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:23.381049 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:23.381049 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:23.381049 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:23.381049 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:23.381049 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:23.381049 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:23.381049 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 12:02:23.381049 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 12:02:23.381049 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 12:02:23.382690 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 12:02:23.382690 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:23.382690 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:23.382690 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:23.382690 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:23.382690 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:23.382690 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:23.382690 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:23.382690 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:23.382690 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:23.382690 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:23.382690 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:23.382690 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:23.382690 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:23.382690 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:23.382690 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:23.382690 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:23.390172 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 12:02:23.390172 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 12:02:23.390172 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:02:23.390172 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:02:23.390172 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:02:23.390172 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:02:23.390172 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:23.390172 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:23.391021 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 405/822] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:23.391021 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd829580-07ec-4ee3-8eac-7b559d83ba46 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:23.391257 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 418/823] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:02:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:23.396249 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae397b9-e77d-4c10-a46f-3c24c9707af9 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:23.484662 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:23.501294 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd829580-07ec-4ee3-8eac-7b559d83ba46 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:23.520701 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae397b9-e77d-4c10-a46f-3c24c9707af9 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:23.540064 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75c68e2-3211-49f0-a9c1-23ade846b7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:23.540702 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75c68e2-3211-49f0-a9c1-23ade846b7de None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:23.540969 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75c68e2-3211-49f0-a9c1-23ade846b7de None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:23.541247 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75c68e2-3211-49f0-a9c1-23ade846b7de None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:23.550383 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75c68e2-3211-49f0-a9c1-23ade846b7de None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:23.551874 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75c68e2-3211-49f0-a9c1-23ade846b7de None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:23.560853 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa0526-3e38-448c-8512-45274b3acb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:23.561751 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa0526-3e38-448c-8512-45274b3acb12 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:23.562310 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa0526-3e38-448c-8512-45274b3acb12 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:23.562783 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa0526-3e38-448c-8512-45274b3acb12 None admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:23.563709 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaa0526-3e38-448c-8512-45274b3acb12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:23.563814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-b75c68e2-3211-49f0-a9c1-23ade846b7de None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 12:02:23.563814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:23.563814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:23.563814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:23.563814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:23.563814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:23.563814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:23.563814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:23.563814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:23.563814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:23.563814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:23.563814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:23.563814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 12:02:23.563814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 12:02:23.563814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 12:02:23.563814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 12:02:23.565271 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:23.565271 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:23.565271 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:02:23.565271 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:23.565271 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:02:23.565271 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:23.565271 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:23.565271 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:23.565271 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:23.565271 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:23.565271 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:23.565271 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:23.565271 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:23.565271 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:23.565271 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:23.565271 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:23.565271 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:23.565271 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:23.566227 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:23.566227 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:23.566227 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 12:02:23.566227 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 12:02:23.566227 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 12:02:23.566227 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 12:02:23.566227 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 12:02:23.566227 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 12:02:23.566227 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 12:02:23.566227 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:23.566227 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 406/824] 10.222.0.98 () {56 vars in 1117 bytes} [Thu Aug 18 12:02:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:02:23.567689 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaa0526-3e38-448c-8512-45274b3acb12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:23.569455 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaa0526-3e38-448c-8512-45274b3acb12 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:23.571720 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75c68e2-3211-49f0-a9c1-23ade846b7de None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:23.584766 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 419/825] 10.222.0.98 () {56 vars in 1148 bytes} [Thu Aug 18 12:02:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:23.591349 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa0526-3e38-448c-8512-45274b3acb12 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:23.711302 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75c68e2-3211-49f0-a9c1-23ade846b7de None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:23.755052 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa857ed-84c9-42be-995d-af115d757dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:23.755714 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa857ed-84c9-42be-995d-af115d757dd3 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:23.755995 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa857ed-84c9-42be-995d-af115d757dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:23.756265 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa857ed-84c9-42be-995d-af115d757dd3 None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:23.756997 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa857ed-84c9-42be-995d-af115d757dd3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:23.757393 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa857ed-84c9-42be-995d-af115d757dd3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:23.758845 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa857ed-84c9-42be-995d-af115d757dd3 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:23.768412 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:23.789756 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 407/826] 10.222.0.98 () {56 vars in 1149 bytes} [Thu Aug 18 12:02:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:23.789756 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f323a5d-dbe1-4378-b541-7b25978975eb None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:23.789756 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f323a5d-dbe1-4378-b541-7b25978975eb None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:23.789756 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f323a5d-dbe1-4378-b541-7b25978975eb None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:23.789756 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 408/827] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:23.789756 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f323a5d-dbe1-4378-b541-7b25978975eb None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:23.798547 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:23.824892 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa0526-3e38-448c-8512-45274b3acb12 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:23.862920 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4774f43-cf66-4495-b1c1-40ce6591a0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:23.863617 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4774f43-cf66-4495-b1c1-40ce6591a0f0 None admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:23.863894 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4774f43-cf66-4495-b1c1-40ce6591a0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:23.864173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4774f43-cf66-4495-b1c1-40ce6591a0f0 None admin] PATH_INFO: `/v3/projects/6f676a5169464d83af4267db25025728/users/771591fe31cb4e8c9c4320c2a7b2719d/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:23.866092 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4774f43-cf66-4495-b1c1-40ce6591a0f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f676a5169464d83af4267db25025728, user_id=771591fe31cb4e8c9c4320c2a7b2719d, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:23.867536 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4774f43-cf66-4495-b1c1-40ce6591a0f0 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:23.902171 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f323a5d-dbe1-4378-b541-7b25978975eb None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:23.936868 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ae2c3-9b15-4665-9a04-411a8fc13bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:23.937582 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ae2c3-9b15-4665-9a04-411a8fc13bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:23.937930 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ae2c3-9b15-4665-9a04-411a8fc13bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:23.938269 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ae2c3-9b15-4665-9a04-411a8fc13bc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:23.948283 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ae2c3-9b15-4665-9a04-411a8fc13bc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:23.949695 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ae2c3-9b15-4665-9a04-411a8fc13bc2 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:23.964994 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-e19ae2c3-9b15-4665-9a04-411a8fc13bc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:23.964994 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:23.964994 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:23.964994 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:23.964994 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:23.964994 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:23.964994 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:23.964994 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:23.964994 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:23.964994 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:23.964994 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:23.964994 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:23.964994 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:23.964994 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:23.964994 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:23.964994 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:23.966127 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:23.966127 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:23.966127 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:23.966127 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:23.966127 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:23.966127 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:23.966127 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:23.966127 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:23.966127 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:23.966127 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:23.966127 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:23.966127 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:23.966127 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:23.966127 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:23.966127 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:23.966127 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:23.966127 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:23.972479 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:23.972479 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:23.972479 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:23.972479 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 409/828] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:23.972479 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb6428-c74f-44e6-882c-25485e4964b2 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:23.972479 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb6428-c74f-44e6-882c-25485e4964b2 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:23.972479 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb6428-c74f-44e6-882c-25485e4964b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:23.976603 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 420/829] 10.222.0.98 () {58 vars in 1439 bytes} [Thu Aug 18 12:02:23 2022] PUT /identity/v3/projects/6f676a5169464d83af4267db25025728/users/771591fe31cb4e8c9c4320c2a7b2719d/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:02:23.979579 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4774f43-cf66-4495-b1c1-40ce6591a0f0 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:23.992725 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-24cb6428-c74f-44e6-882c-25485e4964b2 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:23.994107 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-24cb6428-c74f-44e6-882c-25485e4964b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:24.062736 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:24.080610 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 410/830] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:23 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:02:24.120324 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4774f43-cf66-4495-b1c1-40ce6591a0f0 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:24.137997 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:24.157084 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7062d7-c59e-46bc-b412-f66f82ae1c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:24.157704 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7062d7-c59e-46bc-b412-f66f82ae1c75 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:24.157988 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7062d7-c59e-46bc-b412-f66f82ae1c75 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:24.158832 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7062d7-c59e-46bc-b412-f66f82ae1c75 None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:24.159514 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7062d7-c59e-46bc-b412-f66f82ae1c75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:24.159831 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7062d7-c59e-46bc-b412-f66f82ae1c75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:24.161288 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7062d7-c59e-46bc-b412-f66f82ae1c75 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:24.175004 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 421/831] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:24.180676 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb6428-c74f-44e6-882c-25485e4964b2 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:24.303223 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb6428-c74f-44e6-882c-25485e4964b2 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:24.339831 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831cb1f-749e-458a-9518-da755d1fd730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:24.340578 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831cb1f-749e-458a-9518-da755d1fd730 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:24.340970 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831cb1f-749e-458a-9518-da755d1fd730 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:24.341346 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831cb1f-749e-458a-9518-da755d1fd730 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:24.348578 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8831cb1f-749e-458a-9518-da755d1fd730 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:24.350162 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8831cb1f-749e-458a-9518-da755d1fd730 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:24.361974 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-8831cb1f-749e-458a-9518-da755d1fd730 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:24.361974 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:24.361974 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:24.361974 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:24.361974 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:24.361974 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:24.361974 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:24.361974 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:24.361974 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:24.361974 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:24.361974 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:24.361974 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:24.361974 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 12:02:24.361974 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 12:02:24.361974 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 12:02:24.363076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 12:02:24.363076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:24.363076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:24.363076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:24.363076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:24.363076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:24.363076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:24.363076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:24.363076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:24.363076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:24.363076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:24.363076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:24.363076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:24.363076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:24.363076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:24.363076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:24.363076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:24.364312 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 12:02:24.364312 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 12:02:24.364312 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:02:24.364312 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:02:24.364312 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:02:24.364312 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:02:24.364312 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:24.364312 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:24.364890 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7062d7-c59e-46bc-b412-f66f82ae1c75 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:24.364985 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 411/832] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:02:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:02:24.371668 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831cb1f-749e-458a-9518-da755d1fd730 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:24.441554 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:24.495712 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7062d7-c59e-46bc-b412-f66f82ae1c75 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:24.513900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831cb1f-749e-458a-9518-da755d1fd730 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:24.545130 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d4cca-920e-46e5-b1ea-9b1df21c0928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:24.546140 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d4cca-920e-46e5-b1ea-9b1df21c0928 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:24.546628 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d4cca-920e-46e5-b1ea-9b1df21c0928 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:24.547130 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d4cca-920e-46e5-b1ea-9b1df21c0928 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:24.548516 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244d4cca-920e-46e5-b1ea-9b1df21c0928 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:24.550676 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244d4cca-920e-46e5-b1ea-9b1df21c0928 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:24.562352 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-244d4cca-920e-46e5-b1ea-9b1df21c0928 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 12:02:24.562352 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:24.562352 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 12:02:24.562352 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 12:02:24.562352 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 12:02:24.562352 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 12:02:24.562352 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 12:02:24.562352 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:24.562352 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 12:02:24.562352 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:24.562352 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 12:02:24.562352 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:24.562352 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:24.562352 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:24.562352 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:24.562352 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:24.563399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:24.563399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:24.563399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:24.563399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:24.563399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:24.563399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:24.563399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:24.563399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 12:02:24.563399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 12:02:24.563399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 12:02:24.563399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 12:02:24.563399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:24.563399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:24.563399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:24.563399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:24.563399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:24.563399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:24.565479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:24.565479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:24.565479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:24.565479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:24.565479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:24.565479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:24.565479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:24.565479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:24.565479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:24.565479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:24.565479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 12:02:24.565479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:24.565479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 12:02:24.565479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:24.565479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 422/833] 10.222.0.98 () {56 vars in 1132 bytes} [Thu Aug 18 12:02:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:24.569881 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d4cca-920e-46e5-b1ea-9b1df21c0928 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:24.572248 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77043300-05a4-45c7-b89f-a943e337cf25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:24.573186 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77043300-05a4-45c7-b89f-a943e337cf25 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:24.573677 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77043300-05a4-45c7-b89f-a943e337cf25 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:24.574411 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77043300-05a4-45c7-b89f-a943e337cf25 None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:24.576357 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77043300-05a4-45c7-b89f-a943e337cf25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:24.576857 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77043300-05a4-45c7-b89f-a943e337cf25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:24.579308 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77043300-05a4-45c7-b89f-a943e337cf25 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:24.605746 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 412/834] 10.222.0.98 () {56 vars in 1148 bytes} [Thu Aug 18 12:02:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:24.608076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77043300-05a4-45c7-b89f-a943e337cf25 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:24.622279 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:24.692841 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d4cca-920e-46e5-b1ea-9b1df21c0928 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:24.724373 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77043300-05a4-45c7-b89f-a943e337cf25 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:24.729880 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e47d1a-1c13-4e60-8b72-99c35368d7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:24.730567 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e47d1a-1c13-4e60-8b72-99c35368d7df None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:24.730926 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e47d1a-1c13-4e60-8b72-99c35368d7df None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:24.731276 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e47d1a-1c13-4e60-8b72-99c35368d7df None admin] PATH_INFO: `/v3/services` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:24.732022 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e47d1a-1c13-4e60-8b72-99c35368d7df None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:24.732398 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e47d1a-1c13-4e60-8b72-99c35368d7df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:24.733844 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e47d1a-1c13-4e60-8b72-99c35368d7df None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:24.735195 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:24.739783 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 423/835] 10.222.0.98 () {56 vars in 1126 bytes} [Thu Aug 18 12:02:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:02:24.745521 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43472885-318d-4bc7-8594-54041ddb48a8 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:24.745805 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43472885-318d-4bc7-8594-54041ddb48a8 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:24.746080 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43472885-318d-4bc7-8594-54041ddb48a8 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:24.751177 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 424/836] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:24.751177 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43472885-318d-4bc7-8594-54041ddb48a8 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:24.770177 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e87f5-232b-4f35-a435-c1525251557d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:24.771080 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e87f5-232b-4f35-a435-c1525251557d None admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:24.771535 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e87f5-232b-4f35-a435-c1525251557d None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:24.771965 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e87f5-232b-4f35-a435-c1525251557d None admin] PATH_INFO: `/v3/projects/6f676a5169464d83af4267db25025728/users/b6880278f7304051b9ef03e927637e74/roles/563e3ed5c6164aae966b64cc302b114a` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:24.786328 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48e87f5-232b-4f35-a435-c1525251557d None admin] RBAC: Authorizing `identity:create_grant(project_id=6f676a5169464d83af4267db25025728, user_id=b6880278f7304051b9ef03e927637e74, role_id=563e3ed5c6164aae966b64cc302b114a)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:24.787809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48e87f5-232b-4f35-a435-c1525251557d None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:24.849038 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 413/837] 10.222.0.98 () {58 vars in 1439 bytes} [Thu Aug 18 12:02:24 2022] PUT /identity/v3/projects/6f676a5169464d83af4267db25025728/users/b6880278f7304051b9ef03e927637e74/roles/563e3ed5c6164aae966b64cc302b114a => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:02:24.850298 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c754771d-4d98-44c3-a609-b9fe77eeb7f0 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:24.850671 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c754771d-4d98-44c3-a609-b9fe77eeb7f0 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:24.850953 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c754771d-4d98-44c3-a609-b9fe77eeb7f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:24.867757 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43472885-318d-4bc7-8594-54041ddb48a8 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:24.871082 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:24.883773 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-c754771d-4d98-44c3-a609-b9fe77eeb7f0 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:24.885059 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-c754771d-4d98-44c3-a609-b9fe77eeb7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:24.922363 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d6805-b83b-4657-9043-0b9fbb9e3d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:24.923162 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d6805-b83b-4657-9043-0b9fbb9e3d29 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:24.923583 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d6805-b83b-4657-9043-0b9fbb9e3d29 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:24.924116 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d6805-b83b-4657-9043-0b9fbb9e3d29 None admin] PATH_INFO: `/v3/services` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:24.924976 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32d6805-b83b-4657-9043-0b9fbb9e3d29 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:24.925394 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32d6805-b83b-4657-9043-0b9fbb9e3d29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:24.926941 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32d6805-b83b-4657-9043-0b9fbb9e3d29 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:24.932755 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 425/838] 10.222.0.98 () {56 vars in 1126 bytes} [Thu Aug 18 12:02:24 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:24.982488 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 414/839] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:24 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:02:25.252648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d6805-b83b-4657-9043-0b9fbb9e3d29 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:25.354626 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d6805-b83b-4657-9043-0b9fbb9e3d29 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:25.390537 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9815cc34-264f-43ec-a5fa-69cc75aa9c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:25.391694 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815cc34-264f-43ec-a5fa-69cc75aa9c07 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:25.392182 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815cc34-264f-43ec-a5fa-69cc75aa9c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:25.393301 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815cc34-264f-43ec-a5fa-69cc75aa9c07 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:25.394105 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9815cc34-264f-43ec-a5fa-69cc75aa9c07 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:25.395782 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9815cc34-264f-43ec-a5fa-69cc75aa9c07 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:25.405618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-9815cc34-264f-43ec-a5fa-69cc75aa9c07 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 12:02:25.405618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:25.405618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 12:02:25.405618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 12:02:25.405618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 12:02:25.405618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 12:02:25.405618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 12:02:25.405618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:25.405618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 12:02:25.405618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:25.405618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 12:02:25.405618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:25.405618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:25.405618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:25.405618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:25.405618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:25.405618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:25.406606 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:25.406606 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:25.406606 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:25.406606 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:25.406606 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:25.406606 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:25.406606 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 12:02:25.406606 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 12:02:25.406606 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 12:02:25.406606 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 12:02:25.406606 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:25.406606 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:25.406606 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:25.406606 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:25.406606 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:25.406606 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:25.407504 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:25.407504 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:25.407504 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:25.407504 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:25.407504 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:25.407504 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:25.407504 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:25.407504 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:25.407504 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:25.407504 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:25.407504 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 12:02:25.407504 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:25.407504 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 12:02:25.407504 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:25.414786 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 426/840] 10.222.0.98 () {56 vars in 1114 bytes} [Thu Aug 18 12:02:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:25.416472 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c754771d-4d98-44c3-a609-b9fe77eeb7f0 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:25.513394 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:25.526814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c754771d-4d98-44c3-a609-b9fe77eeb7f0 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:25.562674 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7196d3-a295-45ad-8833-a6f421e0b698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:25.563300 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7196d3-a295-45ad-8833-a6f421e0b698 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:25.563660 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7196d3-a295-45ad-8833-a6f421e0b698 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:25.563941 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7196d3-a295-45ad-8833-a6f421e0b698 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:25.564655 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7196d3-a295-45ad-8833-a6f421e0b698 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:25.564966 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7196d3-a295-45ad-8833-a6f421e0b698 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:25.566443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7196d3-a295-45ad-8833-a6f421e0b698 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:25.571782 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 415/841] 10.222.0.98 () {56 vars in 1114 bytes} [Thu Aug 18 12:02:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:25.577429 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9815cc34-264f-43ec-a5fa-69cc75aa9c07 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:25.617019 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:25.690362 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9815cc34-264f-43ec-a5fa-69cc75aa9c07 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:25.725738 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05694f-b32e-45e4-aa55-52e3aea2db9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:25.726351 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05694f-b32e-45e4-aa55-52e3aea2db9c None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:25.726637 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05694f-b32e-45e4-aa55-52e3aea2db9c None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:25.726919 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05694f-b32e-45e4-aa55-52e3aea2db9c None admin] PATH_INFO: `/v3/services` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:25.727577 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05694f-b32e-45e4-aa55-52e3aea2db9c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:25.727887 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05694f-b32e-45e4-aa55-52e3aea2db9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:25.729250 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05694f-b32e-45e4-aa55-52e3aea2db9c None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:25.734097 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 427/842] 10.222.0.98 () {56 vars in 1114 bytes} [Thu Aug 18 12:02:25 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:25.987657 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e3a17-33c7-4adf-9b40-6094f63b145e None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:25.988180 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e3a17-33c7-4adf-9b40-6094f63b145e None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:25.988180 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e3a17-33c7-4adf-9b40-6094f63b145e None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:25.989836 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 416/843] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:25.995273 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e1b13-6143-40b2-af5a-1b981d2c3f39 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:25.995775 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e1b13-6143-40b2-af5a-1b981d2c3f39 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:25.995972 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e1b13-6143-40b2-af5a-1b981d2c3f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:26.015767 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-cb0e1b13-6143-40b2-af5a-1b981d2c3f39 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:26.017001 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-cb0e1b13-6143-40b2-af5a-1b981d2c3f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:26.087395 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 428/844] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:25 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:02:26.371138 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e3a17-33c7-4adf-9b40-6094f63b145e None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:26.479053 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e3a17-33c7-4adf-9b40-6094f63b145e None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:26.515060 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb47c2-34a2-4128-96c3-73598c6c1690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:26.515732 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb47c2-34a2-4128-96c3-73598c6c1690 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:26.516016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb47c2-34a2-4128-96c3-73598c6c1690 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:26.516294 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb47c2-34a2-4128-96c3-73598c6c1690 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:26.528429 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebb47c2-34a2-4128-96c3-73598c6c1690 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:02:26.533206 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-3ebb47c2-34a2-4128-96c3-73598c6c1690 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:02:26.533206 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:26.533206 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:26.533206 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:26.533206 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:26.533206 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:26.533206 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:26.533206 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:26.533206 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:26.533206 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:26.533206 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:26.533206 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:26.533206 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:02:26.533206 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:02:26.533206 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:02:26.533206 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:02:26.534820 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:02:26.534820 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:02:26.534820 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:02:26.534820 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:02:26.534820 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:26.534820 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:26.534820 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:26.534820 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:26.534820 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:26.534820 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:26.534820 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:26.534820 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:26.534820 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:26.534820 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:26.534820 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:26.534820 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:26.537042 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:26.537042 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:26.537042 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:26.537042 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:26.537042 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:02:26.537042 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:02:26.537042 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:02:26.537042 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:02:26.537042 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:02:26.537042 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:02:26.537042 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:02:26.537042 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:26.537042 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 417/845] 10.222.0.98 () {56 vars in 1096 bytes} [Thu Aug 18 12:02:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:26.542843 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e1b13-6143-40b2-af5a-1b981d2c3f39 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:26.575869 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:26.652446 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e1b13-6143-40b2-af5a-1b981d2c3f39 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:26.687204 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f9169-0654-4a2f-8bde-2f5a0c63a5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:26.687861 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f9169-0654-4a2f-8bde-2f5a0c63a5a9 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:26.688145 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f9169-0654-4a2f-8bde-2f5a0c63a5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:26.688420 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f9169-0654-4a2f-8bde-2f5a0c63a5a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:26.689092 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f9169-0654-4a2f-8bde-2f5a0c63a5a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:26.689400 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f9169-0654-4a2f-8bde-2f5a0c63a5a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:26.691309 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f9169-0654-4a2f-8bde-2f5a0c63a5a9 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:26.700351 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 429/846] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:02:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:26.711977 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb47c2-34a2-4128-96c3-73598c6c1690 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:26.728742 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:26.824145 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb47c2-34a2-4128-96c3-73598c6c1690 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:26.866743 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e68d7-65e9-4cba-b45d-4c2a7ac74070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:26.867496 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e68d7-65e9-4cba-b45d-4c2a7ac74070 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:26.867787 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e68d7-65e9-4cba-b45d-4c2a7ac74070 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:26.868067 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e68d7-65e9-4cba-b45d-4c2a7ac74070 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:26.879729 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063d85b-0e36-4751-a202-42f37e83eecd None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:26.880001 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063d85b-0e36-4751-a202-42f37e83eecd None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:26.880229 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e68d7-65e9-4cba-b45d-4c2a7ac74070 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:26.880399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063d85b-0e36-4751-a202-42f37e83eecd None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:26.881492 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e68d7-65e9-4cba-b45d-4c2a7ac74070 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:26.881937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 430/847] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 12:02:26.886769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ec999-c6cd-4946-9730-399c61b02cfd None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:26.887060 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ec999-c6cd-4946-9730-399c61b02cfd None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:26.887342 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ec999-c6cd-4946-9730-399c61b02cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:26.899117 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-915e68d7-65e9-4cba-b45d-4c2a7ac74070 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:26.899117 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:26.899117 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:26.899117 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:26.899117 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:26.899117 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:26.899117 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:26.899117 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:26.899117 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:26.899117 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:26.899117 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:26.899117 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:26.899117 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:26.899117 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:26.899117 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:26.899117 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:26.901369 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:26.901369 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:26.901369 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:26.901369 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:26.901369 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:26.901369 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:26.901369 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:26.901369 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:26.901369 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:26.901369 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:26.901369 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:26.901369 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:26.901369 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:26.901369 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:26.901369 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:26.901369 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:26.901369 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:26.902571 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:26.902571 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:26.902571 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:26.902571 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 418/848] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:26.906907 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-b41ec999-c6cd-4946-9730-399c61b02cfd None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:26.909023 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-b41ec999-c6cd-4946-9730-399c61b02cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:26.913136 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e68d7-65e9-4cba-b45d-4c2a7ac74070 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:26.994093 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:27.007482 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 431/849] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:26 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 12:02:27.046843 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e68d7-65e9-4cba-b45d-4c2a7ac74070 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:27.061095 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b724b-7d66-4e37-a1b2-83ff3f938dc0 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:27.061389 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b724b-7d66-4e37-a1b2-83ff3f938dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:27.061671 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b724b-7d66-4e37-a1b2-83ff3f938dc0 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:27.063269 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 432/850] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 12:02:27.068098 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13e4ff-28f3-414f-b2de-08dc9f6c8507 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:27.068386 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13e4ff-28f3-414f-b2de-08dc9f6c8507 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:27.068665 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13e4ff-28f3-414f-b2de-08dc9f6c8507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:27.086837 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-9e13e4ff-28f3-414f-b2de-08dc9f6c8507 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:27.087250 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587eb6ee-f758-4d7b-a5fb-e00ddfa5caf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:27.088066 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-9e13e4ff-28f3-414f-b2de-08dc9f6c8507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:27.088394 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587eb6ee-f758-4d7b-a5fb-e00ddfa5caf6 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:27.088877 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587eb6ee-f758-4d7b-a5fb-e00ddfa5caf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:27.089365 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587eb6ee-f758-4d7b-a5fb-e00ddfa5caf6 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:27.090652 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587eb6ee-f758-4d7b-a5fb-e00ddfa5caf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:27.091830 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587eb6ee-f758-4d7b-a5fb-e00ddfa5caf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:27.093208 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587eb6ee-f758-4d7b-a5fb-e00ddfa5caf6 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:27.103359 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:27.110348 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 419/851] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:27.115811 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587eb6ee-f758-4d7b-a5fb-e00ddfa5caf6 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:27.179638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 433/852] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:27 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 12:02:27.243347 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587eb6ee-f758-4d7b-a5fb-e00ddfa5caf6 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:27.270155 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:27.282451 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bde2527-1203-459a-91b2-68b0fda34ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:27.294642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde2527-1203-459a-91b2-68b0fda34ba1 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:27.294642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde2527-1203-459a-91b2-68b0fda34ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:27.294642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde2527-1203-459a-91b2-68b0fda34ba1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:27.294642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bde2527-1203-459a-91b2-68b0fda34ba1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:27.294959 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13e4ff-28f3-414f-b2de-08dc9f6c8507 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:27.295051 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bde2527-1203-459a-91b2-68b0fda34ba1 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:27.311815 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-1bde2527-1203-459a-91b2-68b0fda34ba1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 18 12:02:27.311815 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:27.311815 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:27.311815 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:27.311815 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:27.311815 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:27.311815 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:27.311815 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:27.311815 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:27.311815 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:27.311815 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:27.311815 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:27.311815 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 12:02:27.311815 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 12:02:27.311815 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 12:02:27.311815 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 12:02:27.312809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:27.312809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:27.312809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:02:27.312809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:27.312809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:02:27.312809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:27.312809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:27.312809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:27.312809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:27.312809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:27.312809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:27.312809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:27.312809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:27.312809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:27.312809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:27.312809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:27.312809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:27.312809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:27.313749 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:27.313749 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:27.313749 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 12:02:27.313749 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 12:02:27.313749 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 12:02:27.313749 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 12:02:27.313749 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 12:02:27.313749 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 12:02:27.313749 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 18 12:02:27.313749 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:27.318127 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 420/853] 10.222.0.98 () {56 vars in 1096 bytes} [Thu Aug 18 12:02:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:27.324013 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bde2527-1203-459a-91b2-68b0fda34ba1 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:27.411326 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13e4ff-28f3-414f-b2de-08dc9f6c8507 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:27.461178 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38af9caf-c412-4441-981f-94d65d3d6f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:27.462103 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bde2527-1203-459a-91b2-68b0fda34ba1 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:27.462422 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af9caf-c412-4441-981f-94d65d3d6f30 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:27.462926 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af9caf-c412-4441-981f-94d65d3d6f30 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:27.463783 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af9caf-c412-4441-981f-94d65d3d6f30 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:27.479565 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38af9caf-c412-4441-981f-94d65d3d6f30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:02:27.486024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-38af9caf-c412-4441-981f-94d65d3d6f30 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:02:27.486024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:27.486024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:27.486024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:27.486024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:27.486024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:27.486024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:27.486024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:27.486024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:27.486024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:27.486024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:27.486024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:27.486024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:02:27.486024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:02:27.486024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:02:27.486024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:02:27.486976 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:02:27.486976 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:02:27.486976 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:02:27.486976 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:02:27.486976 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:27.486976 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:27.486976 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:27.486976 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:27.486976 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:27.486976 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:27.486976 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:27.486976 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:27.486976 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:27.486976 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:27.486976 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:27.486976 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:27.487845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:27.487845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:27.487845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:27.487845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:27.487845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:02:27.487845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:02:27.487845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:02:27.487845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:02:27.487845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:02:27.487845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:02:27.487845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:02:27.487845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:27.488486 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 434/854] 10.222.0.98 () {56 vars in 1102 bytes} [Thu Aug 18 12:02:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:02:27.489097 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38af9caf-c412-4441-981f-94d65d3d6f30 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:27.498621 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:27.500617 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab43ee-e3a6-494b-aaa7-e8cb544308ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:27.501278 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab43ee-e3a6-494b-aaa7-e8cb544308ec None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:27.501650 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab43ee-e3a6-494b-aaa7-e8cb544308ec None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:27.501907 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab43ee-e3a6-494b-aaa7-e8cb544308ec None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:27.503102 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab43ee-e3a6-494b-aaa7-e8cb544308ec None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:27.503448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab43ee-e3a6-494b-aaa7-e8cb544308ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:27.504954 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab43ee-e3a6-494b-aaa7-e8cb544308ec None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:27.530636 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 421/855] 10.222.0.98 () {56 vars in 1135 bytes} [Thu Aug 18 12:02:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:27.531542 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab43ee-e3a6-494b-aaa7-e8cb544308ec None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:27.582952 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:27.599612 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38af9caf-c412-4441-981f-94d65d3d6f30 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:27.636327 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d7c77-ae3b-4239-ad30-2c6a1097f4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:27.636958 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d7c77-ae3b-4239-ad30-2c6a1097f4b1 None admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:27.637301 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d7c77-ae3b-4239-ad30-2c6a1097f4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:27.637515 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d7c77-ae3b-4239-ad30-2c6a1097f4b1 None admin] PATH_INFO: `/v3/services` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:27.638146 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d7c77-ae3b-4239-ad30-2c6a1097f4b1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:27.639567 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d7c77-ae3b-4239-ad30-2c6a1097f4b1 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:27.656051 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab43ee-e3a6-494b-aaa7-e8cb544308ec None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:27.695616 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee50f6-7fd6-4ba6-945c-2ede4b25b38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:27.695738 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee50f6-7fd6-4ba6-945c-2ede4b25b38b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:27.695947 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee50f6-7fd6-4ba6-945c-2ede4b25b38b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:27.696258 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee50f6-7fd6-4ba6-945c-2ede4b25b38b None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:27.696890 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee50f6-7fd6-4ba6-945c-2ede4b25b38b None admin] RBAC: Adding query filter params (name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:27.697197 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee50f6-7fd6-4ba6-945c-2ede4b25b38b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:27.698566 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee50f6-7fd6-4ba6-945c-2ede4b25b38b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:27.707178 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 422/856] 10.222.0.98 () {56 vars in 1103 bytes} [Thu Aug 18 12:02:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:27.708301 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee50f6-7fd6-4ba6-945c-2ede4b25b38b None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:27.723262 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 435/857] 10.222.0.98 () {60 vars in 1141 bytes} [Thu Aug 18 12:02:27 2022] POST /identity/v3/services => generated 269 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:02:27.724717 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585accdb-fec2-47fe-a23e-0e1723f961b5 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:27.725013 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585accdb-fec2-47fe-a23e-0e1723f961b5 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:27.725289 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585accdb-fec2-47fe-a23e-0e1723f961b5 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:27.726509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 436/858] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:27.727234 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585accdb-fec2-47fe-a23e-0e1723f961b5 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:27.806728 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:27.830359 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee50f6-7fd6-4ba6-945c-2ede4b25b38b None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:27.833008 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:27.854898 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585accdb-fec2-47fe-a23e-0e1723f961b5 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:27.870051 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e5fb91-209d-42b2-9c33-2ff1e01706ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:27.870669 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5fb91-209d-42b2-9c33-2ff1e01706ba None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:27.870961 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5fb91-209d-42b2-9c33-2ff1e01706ba None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:27.871242 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5fb91-209d-42b2-9c33-2ff1e01706ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:27.902347 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237a9e9-fc18-4af0-99d1-c4d07391c7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:27.902999 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237a9e9-fc18-4af0-99d1-c4d07391c7c6 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:27.903291 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237a9e9-fc18-4af0-99d1-c4d07391c7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:27.903650 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237a9e9-fc18-4af0-99d1-c4d07391c7c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:27.917333 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d237a9e9-fc18-4af0-99d1-c4d07391c7c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:27.919126 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d237a9e9-fc18-4af0-99d1-c4d07391c7c6 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:27.939212 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-d237a9e9-fc18-4af0-99d1-c4d07391c7c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:27.939212 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:27.939212 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:27.939212 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:27.939212 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:27.939212 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:27.939212 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:27.939212 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:27.939212 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:27.939212 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:27.939212 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:27.939212 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:27.939212 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:27.939212 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:27.939212 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:27.939212 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:27.940855 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:27.940855 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:27.940855 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:27.940855 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:27.940855 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:27.940855 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:27.940855 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:27.940855 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:27.940855 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:27.940855 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:27.940855 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:27.940855 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:27.940855 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:27.940855 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:27.940855 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:27.940855 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:27.940855 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:27.941753 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:27.941753 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:27.941753 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:27.943525 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 437/859] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:27.945082 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4280c7-c182-4a5a-93a7-fab11167620d None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:27.945583 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4280c7-c182-4a5a-93a7-fab11167620d None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:27.951778 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4280c7-c182-4a5a-93a7-fab11167620d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:27.956366 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e5fb91-209d-42b2-9c33-2ff1e01706ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:27.957840 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e5fb91-209d-42b2-9c33-2ff1e01706ba None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:27.968657 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-4e4280c7-c182-4a5a-93a7-fab11167620d None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:27.970971 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-4e4280c7-c182-4a5a-93a7-fab11167620d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:28.046197 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 438/860] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:27 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:02:28.046995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4280c7-c182-4a5a-93a7-fab11167620d None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:28.076717 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 423/861] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:02:27 2022] GET /identity/v3/auth/tokens => generated 1271 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 12:02:28.082438 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e5fb91-209d-42b2-9c33-2ff1e01706ba None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:28.086693 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:28.115601 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:28.193011 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4280c7-c182-4a5a-93a7-fab11167620d None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:28.217505 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e5fb91-209d-42b2-9c33-2ff1e01706ba None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:28.239486 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e4e6c-72f9-40eb-b77e-a8379d781011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:28.240700 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e4e6c-72f9-40eb-b77e-a8379d781011 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:28.241184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e4e6c-72f9-40eb-b77e-a8379d781011 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:28.242008 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e4e6c-72f9-40eb-b77e-a8379d781011 None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:28.242872 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0e4e6c-72f9-40eb-b77e-a8379d781011 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:28.243363 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0e4e6c-72f9-40eb-b77e-a8379d781011 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:28.245112 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0e4e6c-72f9-40eb-b77e-a8379d781011 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:28.262610 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 439/862] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:28.270360 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e4e6c-72f9-40eb-b77e-a8379d781011 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:28.274574 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb5a30-74c9-477e-8c4e-198b599cf3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:28.275220 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb5a30-74c9-477e-8c4e-198b599cf3c0 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:28.275539 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb5a30-74c9-477e-8c4e-198b599cf3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:28.275857 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb5a30-74c9-477e-8c4e-198b599cf3c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:28.290284 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcb5a30-74c9-477e-8c4e-198b599cf3c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:28.292271 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcb5a30-74c9-477e-8c4e-198b599cf3c0 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:28.312076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-fbcb5a30-74c9-477e-8c4e-198b599cf3c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:28.312076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:28.312076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:28.312076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:28.312076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:28.312076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:28.312076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:28.312076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:28.312076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:28.312076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:28.312076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:28.312076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:28.312076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:28.312076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:28.312076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:28.312076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:28.313009 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:28.313009 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:28.313009 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:28.313009 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:28.313009 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:28.313009 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:28.313009 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:28.313009 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:28.313009 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:28.313009 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:28.313009 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:28.313009 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:28.313009 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:28.313009 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:28.313009 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:28.313009 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:28.313009 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:28.313892 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:28.313892 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:28.313892 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:28.315507 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 424/863] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:02:28.319047 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb5a30-74c9-477e-8c4e-198b599cf3c0 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:28.378123 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e4e6c-72f9-40eb-b77e-a8379d781011 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:28.380476 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:28.391898 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:28.412901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a10456-c7d8-4846-adea-8b93c7b8ef81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:28.413790 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a10456-c7d8-4846-adea-8b93c7b8ef81 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:28.414525 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a10456-c7d8-4846-adea-8b93c7b8ef81 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:28.414808 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a10456-c7d8-4846-adea-8b93c7b8ef81 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:28.422813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a10456-c7d8-4846-adea-8b93c7b8ef81 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:28.424912 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a10456-c7d8-4846-adea-8b93c7b8ef81 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:28.436271 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-56a10456-c7d8-4846-adea-8b93c7b8ef81 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 12:02:28.436271 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:28.436271 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:28.436271 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:28.436271 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:28.436271 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:28.436271 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:28.436271 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:28.436271 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:28.436271 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:28.436271 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:28.436271 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:28.436271 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 12:02:28.436271 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 12:02:28.436271 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 12:02:28.436271 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 12:02:28.437218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:28.437218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:28.437218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:02:28.437218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:28.437218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:02:28.437218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:28.437218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:28.437218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:28.437218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:28.437218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:28.437218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:28.437218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:28.437218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:28.437218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:28.437218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:28.437218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:28.437218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:28.437218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:28.438226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:28.438226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:28.438226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 12:02:28.438226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 12:02:28.438226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 12:02:28.438226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 12:02:28.438226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 12:02:28.438226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 12:02:28.438226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 12:02:28.438226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:28.438226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 440/864] 10.222.0.98 () {56 vars in 1117 bytes} [Thu Aug 18 12:02:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:02:28.438820 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a10456-c7d8-4846-adea-8b93c7b8ef81 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:28.442295 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb5a30-74c9-477e-8c4e-198b599cf3c0 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:28.479762 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd90684-ab37-4ff7-a713-5b58730f9aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:28.480375 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd90684-ab37-4ff7-a713-5b58730f9aa1 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:28.480660 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd90684-ab37-4ff7-a713-5b58730f9aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:28.480939 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd90684-ab37-4ff7-a713-5b58730f9aa1 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:28.481589 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd90684-ab37-4ff7-a713-5b58730f9aa1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:28.481906 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd90684-ab37-4ff7-a713-5b58730f9aa1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:28.483350 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd90684-ab37-4ff7-a713-5b58730f9aa1 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:28.499737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 425/865] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:02:28.499737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd90684-ab37-4ff7-a713-5b58730f9aa1 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:28.547728 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a10456-c7d8-4846-adea-8b93c7b8ef81 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:28.595013 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7323be-1811-487d-905b-deaeb06e6671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:28.596681 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7323be-1811-487d-905b-deaeb06e6671 None admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:28.596979 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7323be-1811-487d-905b-deaeb06e6671 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:28.597549 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7323be-1811-487d-905b-deaeb06e6671 None admin] PATH_INFO: `/v3/services` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:28.597913 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7323be-1811-487d-905b-deaeb06e6671 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:28.599286 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7323be-1811-487d-905b-deaeb06e6671 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:28.621509 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd90684-ab37-4ff7-a713-5b58730f9aa1 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:28.638698 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:28.660918 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773633d5-b4ba-444a-9f44-1af59fbccc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:28.661532 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773633d5-b4ba-444a-9f44-1af59fbccc5f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:28.661822 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773633d5-b4ba-444a-9f44-1af59fbccc5f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:28.667635 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773633d5-b4ba-444a-9f44-1af59fbccc5f None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:28.667858 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 441/866] 10.222.0.98 () {60 vars in 1141 bytes} [Thu Aug 18 12:02:28 2022] POST /identity/v3/services => generated 251 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:02:28.667945 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773633d5-b4ba-444a-9f44-1af59fbccc5f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:28.667945 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773633d5-b4ba-444a-9f44-1af59fbccc5f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:28.668090 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7323be-1811-487d-905b-deaeb06e6671 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:28.668162 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773633d5-b4ba-444a-9f44-1af59fbccc5f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:28.687704 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 426/867] 10.222.0.98 () {56 vars in 1149 bytes} [Thu Aug 18 12:02:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:28.695754 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773633d5-b4ba-444a-9f44-1af59fbccc5f None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:28.720611 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:28.831152 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7323be-1811-487d-905b-deaeb06e6671 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:28.835798 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773633d5-b4ba-444a-9f44-1af59fbccc5f None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:28.884163 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda6cf91-813d-4857-9aa9-c861ae4e8143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:28.884864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6cf91-813d-4857-9aa9-c861ae4e8143 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:28.885150 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6cf91-813d-4857-9aa9-c861ae4e8143 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:28.885428 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6cf91-813d-4857-9aa9-c861ae4e8143 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:28.885792 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabbc59f-d90c-459e-9421-94fb9ca980e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:28.886590 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabbc59f-d90c-459e-9421-94fb9ca980e3 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:28.886872 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabbc59f-d90c-459e-9421-94fb9ca980e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:28.887221 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabbc59f-d90c-459e-9421-94fb9ca980e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:28.896521 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda6cf91-813d-4857-9aa9-c861ae4e8143 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:28.898566 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda6cf91-813d-4857-9aa9-c861ae4e8143 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:28.914907 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-eda6cf91-813d-4857-9aa9-c861ae4e8143 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:28.914907 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:28.914907 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:28.914907 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:28.914907 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:28.914907 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:28.914907 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:28.914907 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:28.914907 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:28.914907 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:28.914907 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:28.914907 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:28.914907 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 12:02:28.914907 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 12:02:28.914907 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 12:02:28.915938 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 12:02:28.915938 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:28.915938 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:28.915938 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:28.915938 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:28.915938 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:28.915938 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:28.915938 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:28.915938 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:28.915938 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:28.915938 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:28.915938 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:28.915938 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:28.915938 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:28.915938 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:28.915938 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:28.915938 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:28.916847 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 12:02:28.916847 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 12:02:28.916847 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:02:28.916847 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:02:28.916847 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:02:28.916847 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:02:28.916847 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:28.916847 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:28.917615 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 442/868] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:02:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:28.923508 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda6cf91-813d-4857-9aa9-c861ae4e8143 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:28.937262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:28.964473 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabbc59f-d90c-459e-9421-94fb9ca980e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:28.966202 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabbc59f-d90c-459e-9421-94fb9ca980e3 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:29.043807 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:29.053006 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda6cf91-813d-4857-9aa9-c861ae4e8143 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:29.078891 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 427/869] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:02:28 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 12:02:29.084079 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabbc59f-d90c-459e-9421-94fb9ca980e3 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:29.092484 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41aaad8-e1e0-4892-8072-77616ba94263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:29.093582 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41aaad8-e1e0-4892-8072-77616ba94263 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:29.093863 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41aaad8-e1e0-4892-8072-77616ba94263 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:29.094139 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41aaad8-e1e0-4892-8072-77616ba94263 None admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:29.095242 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41aaad8-e1e0-4892-8072-77616ba94263 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:29.095782 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41aaad8-e1e0-4892-8072-77616ba94263 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:29.098434 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41aaad8-e1e0-4892-8072-77616ba94263 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:29.115254 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 443/870] 10.222.0.98 () {56 vars in 1148 bytes} [Thu Aug 18 12:02:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:02:29.124459 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41aaad8-e1e0-4892-8072-77616ba94263 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:29.169474 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:29.195714 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabbc59f-d90c-459e-9421-94fb9ca980e3 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:29.233705 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad3a7e6-c783-4a38-8514-499de5250372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:29.234731 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3a7e6-c783-4a38-8514-499de5250372 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:29.235214 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3a7e6-c783-4a38-8514-499de5250372 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:29.235683 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3a7e6-c783-4a38-8514-499de5250372 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:29.245577 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41aaad8-e1e0-4892-8072-77616ba94263 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:29.247484 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad3a7e6-c783-4a38-8514-499de5250372 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:29.249267 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad3a7e6-c783-4a38-8514-499de5250372 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:29.266128 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-0ad3a7e6-c783-4a38-8514-499de5250372 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:29.266128 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:29.266128 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:29.266128 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:29.266128 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:29.266128 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:29.266128 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:29.266128 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:29.266128 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:29.266128 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:29.266128 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:29.266128 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:29.266128 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:29.266128 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:29.266128 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:29.266128 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:29.267147 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:29.267147 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:29.267147 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:29.267147 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:29.267147 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:29.267147 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:29.267147 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:29.267147 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:29.267147 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:29.267147 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:29.267147 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:29.267147 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:29.267147 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:29.267147 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:29.267147 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:29.267147 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:29.267147 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:29.268584 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:29.268584 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:29.268584 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:29.272056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 428/871] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:29.277415 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad3a7e6-c783-4a38-8514-499de5250372 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:29.290859 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b303b7f0-365e-41a9-a79c-2455e76903f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:29.292477 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303b7f0-365e-41a9-a79c-2455e76903f2 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:29.292998 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303b7f0-365e-41a9-a79c-2455e76903f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:29.293475 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303b7f0-365e-41a9-a79c-2455e76903f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:29.294943 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b303b7f0-365e-41a9-a79c-2455e76903f2 None admin] RBAC: Adding query filter params (role.id=c53a0e13b58642f291c856efe1f0b87d, scope.project.id=6f676a5169464d83af4267db25025728, user.id=771591fe31cb4e8c9c4320c2a7b2719d) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:29.295893 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b303b7f0-365e-41a9-a79c-2455e76903f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:29.297907 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b303b7f0-365e-41a9-a79c-2455e76903f2 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:29.312048 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 444/872] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:02:29 2022] GET /identity/v3/role_assignments?user.id=771591fe31cb4e8c9c4320c2a7b2719d&scope.project.id=6f676a5169464d83af4267db25025728&role.id=c53a0e13b58642f291c856efe1f0b87d => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:29.406880 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad3a7e6-c783-4a38-8514-499de5250372 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:29.435109 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:29.449994 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b4390-e5fc-432b-bd7b-64a5bb367dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:29.454075 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b4390-e5fc-432b-bd7b-64a5bb367dca None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:29.454075 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b4390-e5fc-432b-bd7b-64a5bb367dca None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:29.454075 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b4390-e5fc-432b-bd7b-64a5bb367dca None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:29.454075 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49b4390-e5fc-432b-bd7b-64a5bb367dca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:29.454075 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49b4390-e5fc-432b-bd7b-64a5bb367dca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:29.454075 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49b4390-e5fc-432b-bd7b-64a5bb367dca None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:29.468317 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 429/873] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:29.476260 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b303b7f0-365e-41a9-a79c-2455e76903f2 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:29.478208 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:29.583618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b303b7f0-365e-41a9-a79c-2455e76903f2 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:29.619871 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31db66be-55a6-4fee-8a05-ae6f06759fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:29.620867 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db66be-55a6-4fee-8a05-ae6f06759fc1 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:29.621364 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db66be-55a6-4fee-8a05-ae6f06759fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:29.621848 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db66be-55a6-4fee-8a05-ae6f06759fc1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:29.629280 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31db66be-55a6-4fee-8a05-ae6f06759fc1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:29.631019 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31db66be-55a6-4fee-8a05-ae6f06759fc1 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:29.642621 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-31db66be-55a6-4fee-8a05-ae6f06759fc1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:29.642621 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:29.642621 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:29.642621 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:29.642621 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:29.642621 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:29.642621 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:29.642621 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:29.642621 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:29.642621 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:29.642621 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:29.642621 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:29.642621 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 12:02:29.642621 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 12:02:29.642621 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 12:02:29.643596 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 12:02:29.643596 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:29.643596 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:29.643596 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:29.643596 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:29.643596 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:29.643596 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:29.643596 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:29.643596 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:29.643596 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:29.643596 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:29.643596 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:29.643596 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:29.643596 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:29.643596 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:29.643596 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:29.643596 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:29.644537 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 12:02:29.644537 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 12:02:29.644537 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:02:29.644537 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:02:29.644537 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:02:29.644537 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:02:29.644537 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:29.644537 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:29.646845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 445/874] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:02:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:29.653270 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b4390-e5fc-432b-bd7b-64a5bb367dca None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:29.752168 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7184878-161e-45d5-9bc6-a3d1e641433c None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:29.752568 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7184878-161e-45d5-9bc6-a3d1e641433c None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:29.752766 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7184878-161e-45d5-9bc6-a3d1e641433c None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:29.754367 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 446/875] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:29.759928 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211070d-506a-4934-8c35-0f1b1d06e38f None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:29.760203 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211070d-506a-4934-8c35-0f1b1d06e38f None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:29.760479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211070d-506a-4934-8c35-0f1b1d06e38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:29.769022 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b4390-e5fc-432b-bd7b-64a5bb367dca None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:29.779072 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-d211070d-506a-4934-8c35-0f1b1d06e38f None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:29.780311 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-d211070d-506a-4934-8c35-0f1b1d06e38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:29.816960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159598b4-03f4-4590-85d6-d75c35ab9e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:29.817843 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159598b4-03f4-4590-85d6-d75c35ab9e8b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:29.819029 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159598b4-03f4-4590-85d6-d75c35ab9e8b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:29.819321 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159598b4-03f4-4590-85d6-d75c35ab9e8b None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:29.820074 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159598b4-03f4-4590-85d6-d75c35ab9e8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:29.820388 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159598b4-03f4-4590-85d6-d75c35ab9e8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:29.821827 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159598b4-03f4-4590-85d6-d75c35ab9e8b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:29.840721 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 430/876] 10.222.0.98 () {56 vars in 1148 bytes} [Thu Aug 18 12:02:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:29.848120 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159598b4-03f4-4590-85d6-d75c35ab9e8b None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:29.854749 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:29.867530 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 447/877] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:29 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:02:29.902810 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:29.962120 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159598b4-03f4-4590-85d6-d75c35ab9e8b None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:29.997336 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259500b3-42d0-49d5-b675-a42c8e5715d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:29.998186 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259500b3-42d0-49d5-b675-a42c8e5715d5 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:29.998638 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259500b3-42d0-49d5-b675-a42c8e5715d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:29.999057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259500b3-42d0-49d5-b675-a42c8e5715d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:29.999965 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259500b3-42d0-49d5-b675-a42c8e5715d5 None admin] RBAC: Adding query filter params (role.id=563e3ed5c6164aae966b64cc302b114a, scope.project.id=6f676a5169464d83af4267db25025728, user.id=b6880278f7304051b9ef03e927637e74) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:30.000419 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259500b3-42d0-49d5-b675-a42c8e5715d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:30.002256 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259500b3-42d0-49d5-b675-a42c8e5715d5 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:30.008950 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 431/878] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:02:29 2022] GET /identity/v3/role_assignments?user.id=b6880278f7304051b9ef03e927637e74&scope.project.id=6f676a5169464d83af4267db25025728&role.id=563e3ed5c6164aae966b64cc302b114a => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:30.145018 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211070d-506a-4934-8c35-0f1b1d06e38f None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:30.251888 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211070d-506a-4934-8c35-0f1b1d06e38f None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:30.293510 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7ec0c-a944-44b9-be04-7264c5a09336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:30.294149 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7ec0c-a944-44b9-be04-7264c5a09336 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:30.294430 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7ec0c-a944-44b9-be04-7264c5a09336 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:30.294706 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7ec0c-a944-44b9-be04-7264c5a09336 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:30.295290 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b7ec0c-a944-44b9-be04-7264c5a09336 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:30.296697 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b7ec0c-a944-44b9-be04-7264c5a09336 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:30.305813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-86b7ec0c-a944-44b9-be04-7264c5a09336 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 12:02:30.305813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:30.305813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 12:02:30.305813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 12:02:30.305813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 12:02:30.305813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 12:02:30.305813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 12:02:30.305813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:30.305813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 12:02:30.305813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:30.305813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 12:02:30.305813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:30.305813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:30.305813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:30.305813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:30.305813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:30.306762 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:30.306762 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:30.306762 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:30.306762 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:30.306762 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:30.306762 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:30.306762 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:30.306762 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 12:02:30.306762 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 12:02:30.306762 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 12:02:30.306762 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 12:02:30.306762 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:30.306762 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:30.306762 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:30.306762 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:30.306762 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:30.306762 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:30.307857 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:30.307857 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:30.307857 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:30.307857 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:30.307857 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:30.307857 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:30.307857 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:30.307857 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:30.307857 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:30.307857 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:30.307857 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 12:02:30.307857 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:30.307857 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 12:02:30.307857 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:30.307857 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 448/879] 10.222.0.98 () {56 vars in 1132 bytes} [Thu Aug 18 12:02:30 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:30.313242 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259500b3-42d0-49d5-b675-a42c8e5715d5 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:30.428338 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259500b3-42d0-49d5-b675-a42c8e5715d5 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:30.439438 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:30.465893 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a2b4c-d96a-4925-923f-5377bf023220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:30.466511 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a2b4c-d96a-4925-923f-5377bf023220 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:30.466791 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a2b4c-d96a-4925-923f-5377bf023220 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:30.467062 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a2b4c-d96a-4925-923f-5377bf023220 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:30.467731 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94a2b4c-d96a-4925-923f-5377bf023220 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:30.468080 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94a2b4c-d96a-4925-923f-5377bf023220 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:30.469444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94a2b4c-d96a-4925-923f-5377bf023220 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:30.475704 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 432/880] 10.222.0.98 () {56 vars in 1126 bytes} [Thu Aug 18 12:02:30 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:30.481305 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7ec0c-a944-44b9-be04-7264c5a09336 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:30.546875 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:30.594622 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7ec0c-a944-44b9-be04-7264c5a09336 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:30.634912 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af55b6f-d2be-4de7-a678-a7de0d3cd7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:30.636013 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af55b6f-d2be-4de7-a678-a7de0d3cd7d4 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:30.636503 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af55b6f-d2be-4de7-a678-a7de0d3cd7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:30.637041 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af55b6f-d2be-4de7-a678-a7de0d3cd7d4 None admin] PATH_INFO: `/v3/services` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:30.637876 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af55b6f-d2be-4de7-a678-a7de0d3cd7d4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:30.638382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af55b6f-d2be-4de7-a678-a7de0d3cd7d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:30.639990 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af55b6f-d2be-4de7-a678-a7de0d3cd7d4 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:30.646433 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 449/881] 10.222.0.98 () {56 vars in 1126 bytes} [Thu Aug 18 12:02:30 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:30.652222 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a2b4c-d96a-4925-923f-5377bf023220 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:30.688257 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad189826-d91e-4f5b-b7ec-78daf5717be6 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:30.688939 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad189826-d91e-4f5b-b7ec-78daf5717be6 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:30.689440 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad189826-d91e-4f5b-b7ec-78daf5717be6 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:30.691915 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 450/882] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 12:02:30.699888 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47265d-beee-4740-b352-86d87f3347d1 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:30.700504 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47265d-beee-4740-b352-86d87f3347d1 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:30.700985 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47265d-beee-4740-b352-86d87f3347d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:30.719801 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-3e47265d-beee-4740-b352-86d87f3347d1 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:30.721421 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-3e47265d-beee-4740-b352-86d87f3347d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:30.765141 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a2b4c-d96a-4925-923f-5377bf023220 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:30.797518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 451/883] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:30 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 12:02:30.807460 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7ac1c-db9a-4122-ba60-7fb66fc47e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:30.807460 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7ac1c-db9a-4122-ba60-7fb66fc47e73 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:30.807460 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7ac1c-db9a-4122-ba60-7fb66fc47e73 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:30.807460 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7ac1c-db9a-4122-ba60-7fb66fc47e73 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:30.807845 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e7ac1c-db9a-4122-ba60-7fb66fc47e73 None admin] RBAC: Adding query filter params (interface=public, service_id=b9deb458a6ea4e259a84c2781a321d80, region_id=RegionOne) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:30.807845 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e7ac1c-db9a-4122-ba60-7fb66fc47e73 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:30.807845 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e7ac1c-db9a-4122-ba60-7fb66fc47e73 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:30.813336 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 433/884] 10.222.0.98 () {56 vars in 1251 bytes} [Thu Aug 18 12:02:30 2022] GET /identity/v3/endpoints?service_id=b9deb458a6ea4e259a84c2781a321d80&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:30.819312 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47265d-beee-4740-b352-86d87f3347d1 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:30.875449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:30.925205 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47265d-beee-4740-b352-86d87f3347d1 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:30.962409 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f0649-c1c0-4744-8ae3-5731d250f725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:30.963024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f0649-c1c0-4744-8ae3-5731d250f725 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:30.963306 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f0649-c1c0-4744-8ae3-5731d250f725 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:30.963611 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f0649-c1c0-4744-8ae3-5731d250f725 None admin] PATH_INFO: `/v3/services` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:30.964219 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f0649-c1c0-4744-8ae3-5731d250f725 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:30.965589 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f0649-c1c0-4744-8ae3-5731d250f725 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:31.026982 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 452/885] 10.222.0.98 () {56 vars in 1087 bytes} [Thu Aug 18 12:02:30 2022] GET /identity/v3/services => generated 1268 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:02:31.066282 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7ac1c-db9a-4122-ba60-7fb66fc47e73 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:31.112759 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:31.180197 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7ac1c-db9a-4122-ba60-7fb66fc47e73 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:31.215887 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a21a38a-d569-4b13-a4bd-a8c8d656c879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:31.216616 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21a38a-d569-4b13-a4bd-a8c8d656c879 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:31.216900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21a38a-d569-4b13-a4bd-a8c8d656c879 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:31.217174 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21a38a-d569-4b13-a4bd-a8c8d656c879 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:31.217759 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a21a38a-d569-4b13-a4bd-a8c8d656c879 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:31.219121 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a21a38a-d569-4b13-a4bd-a8c8d656c879 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:31.228207 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-9a21a38a-d569-4b13-a4bd-a8c8d656c879 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 12:02:31.228207 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:31.228207 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 12:02:31.228207 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 12:02:31.228207 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 12:02:31.228207 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 12:02:31.228207 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 12:02:31.228207 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:31.228207 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 12:02:31.228207 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:31.228207 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 12:02:31.228207 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:31.228207 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:31.228207 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:31.228207 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:31.228207 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:31.228207 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:31.229262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:31.229262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:31.229262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:31.229262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:31.229262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:31.229262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:31.229262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 12:02:31.229262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 12:02:31.229262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 12:02:31.229262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 12:02:31.229262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:31.229262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:31.229262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:31.229262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:31.229262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:31.229262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:31.230356 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:31.230356 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:31.230356 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:31.230356 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:31.230356 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:31.230356 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:31.230356 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:31.230356 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:31.230356 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:31.230356 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:31.230356 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 12:02:31.230356 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:31.230356 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 12:02:31.230356 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:31.230356 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 434/886] 10.222.0.98 () {56 vars in 1114 bytes} [Thu Aug 18 12:02:31 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:31.235485 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f0649-c1c0-4744-8ae3-5731d250f725 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:31.345948 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f0649-c1c0-4744-8ae3-5731d250f725 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:31.363631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:31.382804 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0825c23c-3e2b-47e4-b355-ce201763305c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:31.383549 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825c23c-3e2b-47e4-b355-ce201763305c None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:31.383889 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825c23c-3e2b-47e4-b355-ce201763305c None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:31.384170 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825c23c-3e2b-47e4-b355-ce201763305c None admin] PATH_INFO: `/v3/services` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:31.384804 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0825c23c-3e2b-47e4-b355-ce201763305c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:31.385110 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0825c23c-3e2b-47e4-b355-ce201763305c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:31.386491 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0825c23c-3e2b-47e4-b355-ce201763305c None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:31.391839 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 453/887] 10.222.0.98 () {56 vars in 1114 bytes} [Thu Aug 18 12:02:31 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:31.397983 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a21a38a-d569-4b13-a4bd-a8c8d656c879 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:31.426751 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde797a-c02a-44d0-b4b3-02f05909fba7 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:31.427092 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde797a-c02a-44d0-b4b3-02f05909fba7 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:31.427313 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde797a-c02a-44d0-b4b3-02f05909fba7 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:31.428780 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 454/888] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:31.433888 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bafbb4-bb89-4b2a-8c6d-2107540c8d5b None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:31.434179 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bafbb4-bb89-4b2a-8c6d-2107540c8d5b None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:31.434436 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bafbb4-bb89-4b2a-8c6d-2107540c8d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:31.454404 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-46bafbb4-bb89-4b2a-8c6d-2107540c8d5b None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:31.456386 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-46bafbb4-bb89-4b2a-8c6d-2107540c8d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:31.519398 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a21a38a-d569-4b13-a4bd-a8c8d656c879 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:31.521495 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:31.531087 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 455/889] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:31 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 12:02:31.560919 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf057aa3-64e0-4435-b5fd-a692568e221d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:31.561531 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf057aa3-64e0-4435-b5fd-a692568e221d None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:31.561820 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf057aa3-64e0-4435-b5fd-a692568e221d None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:31.562123 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf057aa3-64e0-4435-b5fd-a692568e221d None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:31.562735 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf057aa3-64e0-4435-b5fd-a692568e221d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:31.563046 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf057aa3-64e0-4435-b5fd-a692568e221d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:31.564445 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf057aa3-64e0-4435-b5fd-a692568e221d None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:31.570232 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 435/890] 10.222.0.98 () {56 vars in 1114 bytes} [Thu Aug 18 12:02:31 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:31.575712 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bafbb4-bb89-4b2a-8c6d-2107540c8d5b None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:31.683076 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bafbb4-bb89-4b2a-8c6d-2107540c8d5b None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:31.694613 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:31.719358 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181b582-d4bc-4f1c-8668-b6ad331d2db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:31.720329 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181b582-d4bc-4f1c-8668-b6ad331d2db2 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:31.720768 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181b582-d4bc-4f1c-8668-b6ad331d2db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:31.721051 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181b582-d4bc-4f1c-8668-b6ad331d2db2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:31.722317 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181b582-d4bc-4f1c-8668-b6ad331d2db2 None admin] RBAC: Adding query filter params (interface=public, service_id=8a5d8b5524d848ff9cd895d1ea860cfb, region_id=RegionOne) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:31.722690 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181b582-d4bc-4f1c-8668-b6ad331d2db2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:31.724153 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181b582-d4bc-4f1c-8668-b6ad331d2db2 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:31.729949 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 456/891] 10.222.0.98 () {56 vars in 1251 bytes} [Thu Aug 18 12:02:31 2022] GET /identity/v3/endpoints?service_id=8a5d8b5524d848ff9cd895d1ea860cfb&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:31.735220 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf057aa3-64e0-4435-b5fd-a692568e221d None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:31.800187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181b582-d4bc-4f1c-8668-b6ad331d2db2 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:31.861672 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf057aa3-64e0-4435-b5fd-a692568e221d None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:31.907818 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181b582-d4bc-4f1c-8668-b6ad331d2db2 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:31.911967 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469a364-df57-43e9-9562-b235a0ee8b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:31.912598 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469a364-df57-43e9-9562-b235a0ee8b4b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:31.912882 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469a364-df57-43e9-9562-b235a0ee8b4b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:31.913160 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469a364-df57-43e9-9562-b235a0ee8b4b None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:31.913783 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469a364-df57-43e9-9562-b235a0ee8b4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:31.915166 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469a364-df57-43e9-9562-b235a0ee8b4b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:31.924076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 436/892] 10.222.0.98 () {56 vars in 1087 bytes} [Thu Aug 18 12:02:31 2022] GET /identity/v3/services => generated 1268 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:02:31.956561 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a53f3-b468-49a2-9b89-c2ef17e21ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:31.957224 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a53f3-b468-49a2-9b89-c2ef17e21ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:31.957501 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a53f3-b468-49a2-9b89-c2ef17e21ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:31.957802 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a53f3-b468-49a2-9b89-c2ef17e21ba9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:31.958976 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364a53f3-b468-49a2-9b89-c2ef17e21ba9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:31.960545 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364a53f3-b468-49a2-9b89-c2ef17e21ba9 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:31.970472 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-364a53f3-b468-49a2-9b89-c2ef17e21ba9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 12:02:31.970472 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:31.970472 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 12:02:31.970472 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 12:02:31.970472 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 12:02:31.970472 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 12:02:31.970472 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 12:02:31.970472 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:31.970472 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 12:02:31.970472 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:31.970472 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 12:02:31.970472 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:31.970472 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:31.970472 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:31.970472 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:31.970472 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:31.970472 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:31.971669 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:31.971669 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:31.971669 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:31.971669 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:31.971669 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:31.971669 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:31.971669 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 12:02:31.971669 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 12:02:31.971669 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 12:02:31.971669 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 12:02:31.971669 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:31.971669 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:31.971669 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:31.971669 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:31.971669 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:31.971669 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:31.972556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:31.972556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:31.972556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:31.972556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:31.972556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:31.972556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:31.972556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:31.972556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:31.972556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:31.972556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:31.972556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 12:02:31.972556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:31.972556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 12:02:31.972556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:31.972556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 457/893] 10.222.0.98 () {56 vars in 1126 bytes} [Thu Aug 18 12:02:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:31.978313 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469a364-df57-43e9-9562-b235a0ee8b4b None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:32.014084 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:32.099679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469a364-df57-43e9-9562-b235a0ee8b4b None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:32.138040 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66807859-ba31-4f9b-bd8e-05e09bbe0235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:32.138659 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66807859-ba31-4f9b-bd8e-05e09bbe0235 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:32.138936 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66807859-ba31-4f9b-bd8e-05e09bbe0235 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:32.139213 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66807859-ba31-4f9b-bd8e-05e09bbe0235 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:32.140167 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66807859-ba31-4f9b-bd8e-05e09bbe0235 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:32.140288 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66807859-ba31-4f9b-bd8e-05e09bbe0235 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:32.141581 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66807859-ba31-4f9b-bd8e-05e09bbe0235 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:32.150398 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 437/894] 10.222.0.98 () {56 vars in 1122 bytes} [Thu Aug 18 12:02:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:32.153860 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a53f3-b468-49a2-9b89-c2ef17e21ba9 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:32.199395 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:32.262337 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a53f3-b468-49a2-9b89-c2ef17e21ba9 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:32.300879 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02680d-e3a3-4630-8b76-2e3ad123ddd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:32.301489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02680d-e3a3-4630-8b76-2e3ad123ddd4 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:32.301768 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02680d-e3a3-4630-8b76-2e3ad123ddd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:32.302042 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02680d-e3a3-4630-8b76-2e3ad123ddd4 None admin] PATH_INFO: `/v3/services` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:32.302667 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb02680d-e3a3-4630-8b76-2e3ad123ddd4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:32.302968 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb02680d-e3a3-4630-8b76-2e3ad123ddd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:32.304401 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb02680d-e3a3-4630-8b76-2e3ad123ddd4 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:32.309266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50902521-7658-4e5b-8aa8-1cab7144ea9f None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:32.309552 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50902521-7658-4e5b-8aa8-1cab7144ea9f None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:32.309826 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50902521-7658-4e5b-8aa8-1cab7144ea9f None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:32.311293 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 438/895] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:32.312908 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 458/896] 10.222.0.98 () {56 vars in 1122 bytes} [Thu Aug 18 12:02:32 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:32.320690 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f25554-c689-4a77-9e45-2f0ec84d862d None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:32.320690 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f25554-c689-4a77-9e45-2f0ec84d862d None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:32.320690 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f25554-c689-4a77-9e45-2f0ec84d862d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:32.338175 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-56f25554-c689-4a77-9e45-2f0ec84d862d None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:32.339836 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-56f25554-c689-4a77-9e45-2f0ec84d862d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:32.423530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 439/897] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:32 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:02:32.691519 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02680d-e3a3-4630-8b76-2e3ad123ddd4 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:32.798910 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02680d-e3a3-4630-8b76-2e3ad123ddd4 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:32.801472 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:32.834658 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e6913-1e5a-4437-afee-9205c7a6591c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:32.835681 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e6913-1e5a-4437-afee-9205c7a6591c None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:32.836176 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e6913-1e5a-4437-afee-9205c7a6591c None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:32.836659 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e6913-1e5a-4437-afee-9205c7a6591c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:32.848791 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610e6913-1e5a-4437-afee-9205c7a6591c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:02:32.853836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-610e6913-1e5a-4437-afee-9205c7a6591c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 12:02:32.853836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:32.853836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:32.853836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:32.853836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:32.853836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:32.853836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:32.853836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:32.853836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:32.853836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:32.853836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:32.853836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:32.853836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:02:32.853836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:02:32.853836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:02:32.853836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:02:32.854779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:02:32.854779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:02:32.854779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:02:32.854779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:02:32.854779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:32.854779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:32.854779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:32.854779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:32.854779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:32.854779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:32.854779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:32.854779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:32.854779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:32.854779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:32.854779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:32.854779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:32.855674 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:32.855674 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:32.855674 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:32.855674 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:32.855674 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:02:32.855674 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:02:32.855674 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:02:32.855674 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:02:32.855674 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:02:32.855674 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:02:32.855674 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 12:02:32.855674 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:32.863481 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 459/898] 10.222.0.98 () {56 vars in 1120 bytes} [Thu Aug 18 12:02:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:32.865083 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f25554-c689-4a77-9e45-2f0ec84d862d None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:32.890308 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:32.967800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f25554-c689-4a77-9e45-2f0ec84d862d None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:33.008295 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c8ed2e-78ff-4e53-8b55-07edfa50b4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:33.009123 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8ed2e-78ff-4e53-8b55-07edfa50b4a4 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:33.009558 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8ed2e-78ff-4e53-8b55-07edfa50b4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:33.009839 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8ed2e-78ff-4e53-8b55-07edfa50b4a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:33.010507 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c8ed2e-78ff-4e53-8b55-07edfa50b4a4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:33.010844 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c8ed2e-78ff-4e53-8b55-07edfa50b4a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:33.012553 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c8ed2e-78ff-4e53-8b55-07edfa50b4a4 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:33.022353 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 440/899] 10.222.0.98 () {56 vars in 1115 bytes} [Thu Aug 18 12:02:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:33.028010 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e6913-1e5a-4437-afee-9205c7a6591c None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:33.039716 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baeb8e3-d094-4440-a8c6-9cbfbe72e8d4 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:33.040014 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baeb8e3-d094-4440-a8c6-9cbfbe72e8d4 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:33.040277 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baeb8e3-d094-4440-a8c6-9cbfbe72e8d4 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:33.042297 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 441/900] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:33.048521 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c0491-beee-4d57-8eeb-87df58c4d702 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:33.048940 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c0491-beee-4d57-8eeb-87df58c4d702 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:33.049478 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c0491-beee-4d57-8eeb-87df58c4d702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:33.071945 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-0f7c0491-beee-4d57-8eeb-87df58c4d702 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:33.073156 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-0f7c0491-beee-4d57-8eeb-87df58c4d702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:33.139120 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e6913-1e5a-4437-afee-9205c7a6591c None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:33.157724 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 442/901] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:33 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:02:33.179694 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb76a5a-9ecd-4e65-915d-f0909dc2da21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:33.180336 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb76a5a-9ecd-4e65-915d-f0909dc2da21 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:33.180620 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb76a5a-9ecd-4e65-915d-f0909dc2da21 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:33.180896 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb76a5a-9ecd-4e65-915d-f0909dc2da21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:33.193277 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb76a5a-9ecd-4e65-915d-f0909dc2da21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:33.194777 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb76a5a-9ecd-4e65-915d-f0909dc2da21 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:33.210515 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-efb76a5a-9ecd-4e65-915d-f0909dc2da21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:33.210515 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:33.210515 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:33.210515 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:33.210515 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:33.210515 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:33.210515 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:33.210515 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:33.210515 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:33.210515 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:33.210515 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:33.210515 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:33.210515 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:33.210515 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:33.210515 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:33.210515 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:33.212812 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:33.212812 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:33.212812 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:33.212812 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:33.212812 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:33.212812 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:33.212812 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:33.212812 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:33.212812 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:33.212812 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:33.212812 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:33.212812 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:33.212812 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:33.212812 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:33.212812 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:33.212812 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:33.212812 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:33.214965 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:33.214965 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:33.214965 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:33.219507 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 460/902] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:33.219704 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c0491-beee-4d57-8eeb-87df58c4d702 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:33.236704 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:33.326467 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c0491-beee-4d57-8eeb-87df58c4d702 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:33.364253 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe4497-86a4-4b24-bde4-42804bfb2dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:33.364870 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe4497-86a4-4b24-bde4-42804bfb2dab None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:33.365155 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe4497-86a4-4b24-bde4-42804bfb2dab None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:33.365492 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe4497-86a4-4b24-bde4-42804bfb2dab None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:33.366147 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe4497-86a4-4b24-bde4-42804bfb2dab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:33.366454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe4497-86a4-4b24-bde4-42804bfb2dab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:33.367866 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe4497-86a4-4b24-bde4-42804bfb2dab None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:33.384242 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 443/903] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:33.386577 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb76a5a-9ecd-4e65-915d-f0909dc2da21 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:33.407812 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:33.466538 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe4497-86a4-4b24-bde4-42804bfb2dab None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:33.500884 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb76a5a-9ecd-4e65-915d-f0909dc2da21 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:33.549038 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d11c197-e6e3-4c43-80a9-86cb99b40c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:33.550513 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11c197-e6e3-4c43-80a9-86cb99b40c76 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:33.550809 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11c197-e6e3-4c43-80a9-86cb99b40c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:33.551088 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11c197-e6e3-4c43-80a9-86cb99b40c76 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:33.560423 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:33.560594 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d11c197-e6e3-4c43-80a9-86cb99b40c76 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:33.562326 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d11c197-e6e3-4c43-80a9-86cb99b40c76 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:33.577518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-9d11c197-e6e3-4c43-80a9-86cb99b40c76 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 12:02:33.577518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:33.577518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:33.577518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:33.577518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:33.577518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:33.577518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:33.577518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:33.577518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:33.577518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:33.577518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:33.577518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:33.577518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 12:02:33.577518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 12:02:33.577518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 12:02:33.577518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 12:02:33.584277 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:33.584277 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:33.584277 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:02:33.584277 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:33.584277 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:02:33.584277 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:33.584277 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:33.584277 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:33.584277 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:33.584277 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:33.584277 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:33.584277 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:33.584277 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:33.584277 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:33.584277 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:33.584277 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:33.584277 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:33.584277 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:33.585220 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:33.585220 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:33.585220 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 12:02:33.585220 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 12:02:33.585220 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 12:02:33.585220 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 12:02:33.585220 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 12:02:33.585220 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 12:02:33.585220 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 12:02:33.585220 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:33.585220 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 461/904] 10.222.0.98 () {56 vars in 1117 bytes} [Thu Aug 18 12:02:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:33.587539 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d11c197-e6e3-4c43-80a9-86cb99b40c76 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:33.591209 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe4497-86a4-4b24-bde4-42804bfb2dab None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:33.636004 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f914b-a5ad-44cf-8a73-d5a7045f5565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:33.636638 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f914b-a5ad-44cf-8a73-d5a7045f5565 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:33.636922 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f914b-a5ad-44cf-8a73-d5a7045f5565 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:33.637195 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f914b-a5ad-44cf-8a73-d5a7045f5565 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:33.638083 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6f914b-a5ad-44cf-8a73-d5a7045f5565 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:33.639785 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6f914b-a5ad-44cf-8a73-d5a7045f5565 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:33.651178 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-4a6f914b-a5ad-44cf-8a73-d5a7045f5565 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 12:02:33.651178 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:33.651178 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 12:02:33.651178 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 12:02:33.651178 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 12:02:33.651178 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 12:02:33.651178 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 12:02:33.651178 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:33.651178 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 12:02:33.651178 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:33.651178 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 12:02:33.651178 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:33.651178 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:33.651178 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:33.651178 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:33.651178 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:33.652447 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:33.652447 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:33.652447 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:33.652447 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:33.652447 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:33.652447 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:33.652447 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:33.652447 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 12:02:33.652447 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 12:02:33.652447 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 12:02:33.652447 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 12:02:33.652447 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:33.652447 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:33.652447 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:33.652447 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:33.652447 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:33.652447 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:33.658403 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:33.658403 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:33.658403 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:33.658403 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:33.658403 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:33.658403 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:33.658403 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:33.658403 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:33.658403 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:33.658403 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:33.658403 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 12:02:33.658403 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:33.658403 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 18 12:02:33.658403 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:33.658403 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 444/905] 10.222.0.98 () {56 vars in 1132 bytes} [Thu Aug 18 12:02:33 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:02:33.664613 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f914b-a5ad-44cf-8a73-d5a7045f5565 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:33.695803 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:33.723785 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d11c197-e6e3-4c43-80a9-86cb99b40c76 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:33.761335 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370fa043-6358-44b4-ac10-d28a120c80a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:33.762327 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370fa043-6358-44b4-ac10-d28a120c80a2 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:33.762935 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370fa043-6358-44b4-ac10-d28a120c80a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:33.763469 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370fa043-6358-44b4-ac10-d28a120c80a2 None admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:33.764477 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370fa043-6358-44b4-ac10-d28a120c80a2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:33.765000 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370fa043-6358-44b4-ac10-d28a120c80a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:33.766802 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370fa043-6358-44b4-ac10-d28a120c80a2 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:33.782368 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f914b-a5ad-44cf-8a73-d5a7045f5565 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:33.788168 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 462/906] 10.222.0.98 () {56 vars in 1149 bytes} [Thu Aug 18 12:02:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:02:33.799962 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370fa043-6358-44b4-ac10-d28a120c80a2 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:33.821261 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f5c764-d7df-4421-815b-a47d26240cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:33.822443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5c764-d7df-4421-815b-a47d26240cf0 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:33.822905 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5c764-d7df-4421-815b-a47d26240cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:33.823514 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5c764-d7df-4421-815b-a47d26240cf0 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:33.824641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f5c764-d7df-4421-815b-a47d26240cf0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:33.825127 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f5c764-d7df-4421-815b-a47d26240cf0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:33.826665 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f5c764-d7df-4421-815b-a47d26240cf0 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:33.836105 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 445/907] 10.222.0.98 () {56 vars in 1126 bytes} [Thu Aug 18 12:02:33 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:02:33.843463 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f5c764-d7df-4421-815b-a47d26240cf0 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:33.854695 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:33.939999 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370fa043-6358-44b4-ac10-d28a120c80a2 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:33.957168 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f5c764-d7df-4421-815b-a47d26240cf0 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:34.007810 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:34.007810 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc84af-b139-457f-990a-53736719fbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:34.007810 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc84af-b139-457f-990a-53736719fbe1 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:34.007810 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc84af-b139-457f-990a-53736719fbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:34.008712 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc84af-b139-457f-990a-53736719fbe1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:34.008794 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee025d-8bab-4b76-9ad4-d9b34d8c3657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:34.008794 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee025d-8bab-4b76-9ad4-d9b34d8c3657 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:34.008794 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee025d-8bab-4b76-9ad4-d9b34d8c3657 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:34.008794 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee025d-8bab-4b76-9ad4-d9b34d8c3657 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:34.009058 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ee025d-8bab-4b76-9ad4-d9b34d8c3657 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:34.009058 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ee025d-8bab-4b76-9ad4-d9b34d8c3657 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:34.009058 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ee025d-8bab-4b76-9ad4-d9b34d8c3657 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:34.011310 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 446/908] 10.222.0.98 () {56 vars in 1126 bytes} [Thu Aug 18 12:02:33 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:02:34.017846 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee025d-8bab-4b76-9ad4-d9b34d8c3657 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:34.084852 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc84af-b139-457f-990a-53736719fbe1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:34.086293 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc84af-b139-457f-990a-53736719fbe1 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:34.105286 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:34.139822 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee025d-8bab-4b76-9ad4-d9b34d8c3657 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:34.178178 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca2aded-cd8e-40ca-bb53-3adb57c8bbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:34.179065 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca2aded-cd8e-40ca-bb53-3adb57c8bbd1 None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:34.179527 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca2aded-cd8e-40ca-bb53-3adb57c8bbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:34.179950 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca2aded-cd8e-40ca-bb53-3adb57c8bbd1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:34.180765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca2aded-cd8e-40ca-bb53-3adb57c8bbd1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:34.182303 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca2aded-cd8e-40ca-bb53-3adb57c8bbd1 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:34.200500 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 463/909] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:02:33 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 12:02:34.200500 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035bd96-da31-454a-9a7a-914c7ff72dc6 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:34.200500 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035bd96-da31-454a-9a7a-914c7ff72dc6 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:34.200500 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035bd96-da31-454a-9a7a-914c7ff72dc6 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:34.200500 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 464/910] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:34.203801 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3035bd96-da31-454a-9a7a-914c7ff72dc6 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:34.234838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:34.260607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 447/911] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:02:34 2022] POST /identity/v3/endpoints => generated 347 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:02:34.260607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afecee08-06e8-4e3c-9fb5-5dd95266d1d9 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:34.260607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afecee08-06e8-4e3c-9fb5-5dd95266d1d9 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:34.260607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afecee08-06e8-4e3c-9fb5-5dd95266d1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:34.297832 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-afecee08-06e8-4e3c-9fb5-5dd95266d1d9 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:34.299188 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-afecee08-06e8-4e3c-9fb5-5dd95266d1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:34.345834 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3035bd96-da31-454a-9a7a-914c7ff72dc6 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:34.391441 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 448/912] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:34 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:02:34.391441 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f27d5-8e16-454e-a68a-30e03e89bce6 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:34.391441 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f27d5-8e16-454e-a68a-30e03e89bce6 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:34.391441 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f27d5-8e16-454e-a68a-30e03e89bce6 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:34.391441 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 449/913] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:34.391441 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156207e5-5b0a-4aa6-b530-31a1866942bb None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:34.392063 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156207e5-5b0a-4aa6-b530-31a1866942bb None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:34.392063 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156207e5-5b0a-4aa6-b530-31a1866942bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:34.395402 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106582fc-364d-4dba-9145-47c502cfac2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:34.396338 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106582fc-364d-4dba-9145-47c502cfac2f None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:34.396695 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106582fc-364d-4dba-9145-47c502cfac2f None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:34.397016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106582fc-364d-4dba-9145-47c502cfac2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:34.412215 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-156207e5-5b0a-4aa6-b530-31a1866942bb None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:34.413030 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106582fc-364d-4dba-9145-47c502cfac2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:34.413508 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-156207e5-5b0a-4aa6-b530-31a1866942bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:34.414942 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106582fc-364d-4dba-9145-47c502cfac2f None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:34.438294 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-106582fc-364d-4dba-9145-47c502cfac2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:34.438294 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:34.438294 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:34.438294 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:34.438294 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:34.438294 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:34.438294 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:34.438294 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:34.438294 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:34.438294 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:34.438294 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:34.438294 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:34.438294 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:34.438294 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:34.438294 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:34.438294 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:34.439332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:34.439332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:34.439332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:34.439332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:34.439332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:34.439332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:34.439332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:34.439332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:34.439332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:34.439332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:34.439332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:34.439332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:34.439332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:34.439332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:34.439332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:34.439332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:34.439332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:34.440364 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:34.440364 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:34.440364 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:34.440364 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 465/914] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:34.445463 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106582fc-364d-4dba-9145-47c502cfac2f None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:34.481342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:34.500263 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 450/915] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:34 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 12:02:34.555324 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:34.574799 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106582fc-364d-4dba-9145-47c502cfac2f None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:34.616779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e038b-aed7-4fd2-8fa7-8bce65bdc8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:34.617400 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e038b-aed7-4fd2-8fa7-8bce65bdc8f3 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:34.617747 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e038b-aed7-4fd2-8fa7-8bce65bdc8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:34.618031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e038b-aed7-4fd2-8fa7-8bce65bdc8f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:34.618682 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e038b-aed7-4fd2-8fa7-8bce65bdc8f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:34.618992 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e038b-aed7-4fd2-8fa7-8bce65bdc8f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:34.620456 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e038b-aed7-4fd2-8fa7-8bce65bdc8f3 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:34.637540 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 466/916] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:02:34.640702 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156207e5-5b0a-4aa6-b530-31a1866942bb None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:34.669488 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e038b-aed7-4fd2-8fa7-8bce65bdc8f3 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:34.772965 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156207e5-5b0a-4aa6-b530-31a1866942bb None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:34.782950 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e038b-aed7-4fd2-8fa7-8bce65bdc8f3 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:34.815841 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5f9dc0-9dbf-46ad-8c1a-a2001a5d4923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:34.816908 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f9dc0-9dbf-46ad-8c1a-a2001a5d4923 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:34.817401 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f9dc0-9dbf-46ad-8c1a-a2001a5d4923 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:34.817895 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f9dc0-9dbf-46ad-8c1a-a2001a5d4923 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:34.823460 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3299403c-2bbf-4b1f-a46a-2d251393638f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:34.824079 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3299403c-2bbf-4b1f-a46a-2d251393638f None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:34.824366 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3299403c-2bbf-4b1f-a46a-2d251393638f None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:34.824637 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3299403c-2bbf-4b1f-a46a-2d251393638f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:34.825203 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3299403c-2bbf-4b1f-a46a-2d251393638f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:34.825322 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5f9dc0-9dbf-46ad-8c1a-a2001a5d4923 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:34.826556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3299403c-2bbf-4b1f-a46a-2d251393638f None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:34.826655 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5f9dc0-9dbf-46ad-8c1a-a2001a5d4923 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:34.835580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-3299403c-2bbf-4b1f-a46a-2d251393638f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 12:02:34.835580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:34.835580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 12:02:34.835580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 12:02:34.835580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 12:02:34.835580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 12:02:34.835580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 12:02:34.835580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:34.835580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 12:02:34.835580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:34.835580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 12:02:34.835580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:34.835580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:34.835580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:34.835580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:34.835580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:34.835580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:34.836585 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:34.836585 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:34.836585 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:34.836585 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:34.836585 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:34.836585 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:34.836585 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 12:02:34.836585 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 12:02:34.836585 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 12:02:34.836585 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 12:02:34.836585 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:34.836585 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:34.836585 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:34.836585 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:34.836585 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:34.836585 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:34.838071 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:34.838071 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:34.838071 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:34.838071 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:34.838071 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:34.838071 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:34.838071 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:34.838071 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:34.838071 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:34.838071 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:34.838071 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 12:02:34.838071 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:34.838071 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 18 12:02:34.838071 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:34.838071 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 467/917] 10.222.0.98 () {56 vars in 1114 bytes} [Thu Aug 18 12:02:34 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:02:34.838870 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3299403c-2bbf-4b1f-a46a-2d251393638f None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:34.841072 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-4f5f9dc0-9dbf-46ad-8c1a-a2001a5d4923 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:34.841072 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:34.841072 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:34.841072 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:34.841072 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:34.841072 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:34.841072 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:34.841072 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:34.841072 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:34.841072 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:34.841072 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:34.841072 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:34.841072 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 12:02:34.841072 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 12:02:34.841072 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 12:02:34.842060 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 12:02:34.842060 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:34.842060 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:34.842060 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:34.842060 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:34.842060 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:34.842060 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:34.842060 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:34.842060 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:34.842060 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:34.842060 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:34.842060 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:34.842060 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:34.842060 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:34.842060 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:34.842060 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:34.842060 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:34.842943 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 12:02:34.842943 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 12:02:34.842943 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:02:34.842943 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:02:34.842943 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:02:34.842943 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:02:34.842943 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:34.842943 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:34.844705 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:34.845793 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 451/918] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:02:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:34.851749 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5f9dc0-9dbf-46ad-8c1a-a2001a5d4923 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:34.903739 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:34.951682 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3299403c-2bbf-4b1f-a46a-2d251393638f None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:34.961662 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5f9dc0-9dbf-46ad-8c1a-a2001a5d4923 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:34.995894 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe808f9-6f63-484a-99e2-28b4f78079fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:34.996681 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe808f9-6f63-484a-99e2-28b4f78079fd None admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:34.996968 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe808f9-6f63-484a-99e2-28b4f78079fd None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:34.997247 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe808f9-6f63-484a-99e2-28b4f78079fd None admin] PATH_INFO: `/v3/services` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:34.997893 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe808f9-6f63-484a-99e2-28b4f78079fd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:34.999279 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe808f9-6f63-484a-99e2-28b4f78079fd None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:35.004729 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b52708-4b21-459d-ad53-84db53cec803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:35.006991 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b52708-4b21-459d-ad53-84db53cec803 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:35.006991 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b52708-4b21-459d-ad53-84db53cec803 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:35.006991 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b52708-4b21-459d-ad53-84db53cec803 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:35.006991 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b52708-4b21-459d-ad53-84db53cec803 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:35.006991 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b52708-4b21-459d-ad53-84db53cec803 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:35.008243 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b52708-4b21-459d-ad53-84db53cec803 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:35.013185 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 452/919] 10.222.0.98 () {56 vars in 1114 bytes} [Thu Aug 18 12:02:34 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:35.013986 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b52708-4b21-459d-ad53-84db53cec803 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:35.096256 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 468/920] 10.222.0.98 () {60 vars in 1141 bytes} [Thu Aug 18 12:02:34 2022] POST /identity/v3/services => generated 241 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:02:35.097664 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe808f9-6f63-484a-99e2-28b4f78079fd None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:35.125424 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b52708-4b21-459d-ad53-84db53cec803 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:35.136477 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:35.172141 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0e041-b756-4c25-aa18-2bcfcc0afcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:35.173191 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0e041-b756-4c25-aa18-2bcfcc0afcb7 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:35.173694 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0e041-b756-4c25-aa18-2bcfcc0afcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:35.174197 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0e041-b756-4c25-aa18-2bcfcc0afcb7 None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:35.175314 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:35.175983 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc0e041-b756-4c25-aa18-2bcfcc0afcb7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:35.176516 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc0e041-b756-4c25-aa18-2bcfcc0afcb7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:35.178213 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc0e041-b756-4c25-aa18-2bcfcc0afcb7 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:35.192660 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 453/921] 10.222.0.98 () {56 vars in 1148 bytes} [Thu Aug 18 12:02:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:35.203026 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0e041-b756-4c25-aa18-2bcfcc0afcb7 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:35.226911 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe808f9-6f63-484a-99e2-28b4f78079fd None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:35.278303 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c1082a-6d3a-4461-959b-510bb90479cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:35.279102 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c1082a-6d3a-4461-959b-510bb90479cd None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:35.279519 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c1082a-6d3a-4461-959b-510bb90479cd None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:35.279902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c1082a-6d3a-4461-959b-510bb90479cd None admin] PATH_INFO: `/v3/services` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:35.280677 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c1082a-6d3a-4461-959b-510bb90479cd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:35.280985 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c1082a-6d3a-4461-959b-510bb90479cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:35.282368 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c1082a-6d3a-4461-959b-510bb90479cd None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:35.289752 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 469/922] 10.222.0.98 () {56 vars in 1114 bytes} [Thu Aug 18 12:02:35 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:35.297067 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c1082a-6d3a-4461-959b-510bb90479cd None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:35.335743 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0e041-b756-4c25-aa18-2bcfcc0afcb7 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:35.376874 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2057c200-554f-4e38-b1ab-db05e3838571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:35.377402 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057c200-554f-4e38-b1ab-db05e3838571 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:35.377833 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057c200-554f-4e38-b1ab-db05e3838571 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:35.378107 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057c200-554f-4e38-b1ab-db05e3838571 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:35.378943 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2057c200-554f-4e38-b1ab-db05e3838571 None admin] RBAC: Adding query filter params (role.id=9bcab8f185d94480b7fb843825dfb542, scope.project.id=6f676a5169464d83af4267db25025728, user.id=b6880278f7304051b9ef03e927637e74) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:35.379256 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2057c200-554f-4e38-b1ab-db05e3838571 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:35.380864 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2057c200-554f-4e38-b1ab-db05e3838571 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:35.387249 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 454/923] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:02:35 2022] GET /identity/v3/role_assignments?user.id=b6880278f7304051b9ef03e927637e74&scope.project.id=6f676a5169464d83af4267db25025728&role.id=9bcab8f185d94480b7fb843825dfb542 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:35.430957 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c1082a-6d3a-4461-959b-510bb90479cd None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:35.464949 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:35.475036 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ebec4-01fb-4a76-92a5-cc3b48882b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:35.477724 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ebec4-01fb-4a76-92a5-cc3b48882b9d None admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:35.477724 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ebec4-01fb-4a76-92a5-cc3b48882b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:35.477724 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ebec4-01fb-4a76-92a5-cc3b48882b9d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:35.477724 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058ebec4-01fb-4a76-92a5-cc3b48882b9d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:35.478462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058ebec4-01fb-4a76-92a5-cc3b48882b9d None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:35.610973 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 470/924] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:02:35 2022] POST /identity/v3/endpoints => generated 346 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:02:36.332865 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afe8dc-cf3e-421e-b605-f2f7fe6c147d None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:36.333965 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afe8dc-cf3e-421e-b605-f2f7fe6c147d None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:36.334469 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afe8dc-cf3e-421e-b605-f2f7fe6c147d None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:36.336065 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 455/925] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:36.341611 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4853ddfc-5691-4a39-9746-6427cd1574cc None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:36.342182 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4853ddfc-5691-4a39-9746-6427cd1574cc None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:36.342683 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4853ddfc-5691-4a39-9746-6427cd1574cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:36.362155 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-4853ddfc-5691-4a39-9746-6427cd1574cc None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:36.364232 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-4853ddfc-5691-4a39-9746-6427cd1574cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:36.442080 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 471/926] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:02:36.717465 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40afe8dc-cf3e-421e-b605-f2f7fe6c147d None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:36.753667 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:36.832348 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40afe8dc-cf3e-421e-b605-f2f7fe6c147d None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:36.872906 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a995a02-1cdf-4905-b891-180ee854c926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:36.873958 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a995a02-1cdf-4905-b891-180ee854c926 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:36.874487 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a995a02-1cdf-4905-b891-180ee854c926 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:36.874972 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a995a02-1cdf-4905-b891-180ee854c926 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:36.875809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a995a02-1cdf-4905-b891-180ee854c926 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:36.877488 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a995a02-1cdf-4905-b891-180ee854c926 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:36.887583 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-4a995a02-1cdf-4905-b891-180ee854c926 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 12:02:36.887583 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:36.887583 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 12:02:36.887583 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 12:02:36.887583 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 12:02:36.887583 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 12:02:36.887583 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 12:02:36.887583 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:36.887583 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 12:02:36.887583 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:36.887583 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 12:02:36.887583 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:36.887583 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:36.887583 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:36.887583 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:36.887583 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:36.887583 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:36.888658 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:36.888658 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:36.888658 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:36.888658 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:36.888658 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:36.888658 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:36.888658 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 12:02:36.888658 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 12:02:36.888658 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 12:02:36.888658 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 12:02:36.888658 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:36.888658 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:36.888658 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:36.888658 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:36.888658 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:36.888658 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:36.889556 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:36.889556 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:36.889556 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:36.889556 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:36.889556 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:36.889556 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:36.889556 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:36.889556 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:36.889556 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:36.889556 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:36.889556 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 12:02:36.889556 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:36.889556 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 12:02:36.889556 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:36.893094 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 456/927] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:02:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:36.899444 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4853ddfc-5691-4a39-9746-6427cd1574cc None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:37.006798 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4853ddfc-5691-4a39-9746-6427cd1574cc None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:37.019127 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:37.043374 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52696576-9bcb-4af3-88eb-2ee01f3a5e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:37.044026 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52696576-9bcb-4af3-88eb-2ee01f3a5e69 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:37.044315 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52696576-9bcb-4af3-88eb-2ee01f3a5e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:37.044618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52696576-9bcb-4af3-88eb-2ee01f3a5e69 None admin] PATH_INFO: `/v3/services` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:37.045260 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52696576-9bcb-4af3-88eb-2ee01f3a5e69 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:37.045567 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52696576-9bcb-4af3-88eb-2ee01f3a5e69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:37.046940 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52696576-9bcb-4af3-88eb-2ee01f3a5e69 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:37.052406 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 472/928] 10.222.0.98 () {56 vars in 1112 bytes} [Thu Aug 18 12:02:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:37.057768 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a995a02-1cdf-4905-b891-180ee854c926 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:37.157819 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:37.167653 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a995a02-1cdf-4905-b891-180ee854c926 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:37.203647 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065f2f5-e228-4ad2-bdec-fa6f6eaf3d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:37.204267 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065f2f5-e228-4ad2-bdec-fa6f6eaf3d2d None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:37.204548 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065f2f5-e228-4ad2-bdec-fa6f6eaf3d2d None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:37.204824 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065f2f5-e228-4ad2-bdec-fa6f6eaf3d2d None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:37.205472 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065f2f5-e228-4ad2-bdec-fa6f6eaf3d2d None admin] RBAC: Adding query filter params (type=compute) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:37.205778 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065f2f5-e228-4ad2-bdec-fa6f6eaf3d2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:37.207177 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065f2f5-e228-4ad2-bdec-fa6f6eaf3d2d None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:37.212517 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 457/929] 10.222.0.98 () {56 vars in 1112 bytes} [Thu Aug 18 12:02:37 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:37.221172 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35217b4c-da59-43a6-bd5b-246bc6b886e4 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:37.222331 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35217b4c-da59-43a6-bd5b-246bc6b886e4 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:37.222879 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35217b4c-da59-43a6-bd5b-246bc6b886e4 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:37.226113 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 473/930] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:37.231279 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70041122-8d1d-4784-a68e-f866df5fe224 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:37.231599 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70041122-8d1d-4784-a68e-f866df5fe224 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:37.231875 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70041122-8d1d-4784-a68e-f866df5fe224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:37.252317 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-70041122-8d1d-4784-a68e-f866df5fe224 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:37.253577 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-70041122-8d1d-4784-a68e-f866df5fe224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:37.331796 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 458/931] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:02:37.433180 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf39fd-8460-4669-8609-26bccbc2f6a6 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:37.433646 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf39fd-8460-4669-8609-26bccbc2f6a6 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:37.434025 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf39fd-8460-4669-8609-26bccbc2f6a6 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:37.435863 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 474/932] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:37.441814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52495153-18c0-4d8f-9689-b177c46f2393 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:37.442302 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52495153-18c0-4d8f-9689-b177c46f2393 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:37.442388 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52495153-18c0-4d8f-9689-b177c46f2393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:37.460912 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-52495153-18c0-4d8f-9689-b177c46f2393 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:37.462113 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-52495153-18c0-4d8f-9689-b177c46f2393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:37.533639 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 459/933] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:02:37.596690 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0143e-f252-4d71-b7b1-b1bb0f8f67b9 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:37.597229 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0143e-f252-4d71-b7b1-b1bb0f8f67b9 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:37.597229 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0143e-f252-4d71-b7b1-b1bb0f8f67b9 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:37.598763 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 475/934] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:37.603934 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9debe1-c2f6-4692-9620-30fe4e22b9dc None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:37.604224 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9debe1-c2f6-4692-9620-30fe4e22b9dc None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:37.604507 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9debe1-c2f6-4692-9620-30fe4e22b9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:37.616545 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0143e-f252-4d71-b7b1-b1bb0f8f67b9 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:37.624791 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-2d9debe1-c2f6-4692-9620-30fe4e22b9dc None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:37.626537 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-2d9debe1-c2f6-4692-9620-30fe4e22b9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:37.641723 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:37.710073 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 460/935] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:02:37.734212 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0143e-f252-4d71-b7b1-b1bb0f8f67b9 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:37.771331 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0633119-561f-45e0-822f-8acddd8eeb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:37.771996 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0633119-561f-45e0-822f-8acddd8eeb96 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:37.772275 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0633119-561f-45e0-822f-8acddd8eeb96 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:37.772549 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0633119-561f-45e0-822f-8acddd8eeb96 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:37.773132 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0633119-561f-45e0-822f-8acddd8eeb96 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:37.774492 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0633119-561f-45e0-822f-8acddd8eeb96 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:37.783798 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-b0633119-561f-45e0-822f-8acddd8eeb96 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 12:02:37.783798 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:37.783798 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 12:02:37.783798 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 12:02:37.783798 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 12:02:37.783798 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 12:02:37.783798 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 12:02:37.783798 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:37.783798 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 12:02:37.783798 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:37.783798 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 12:02:37.783798 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:37.783798 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:37.783798 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:37.783798 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:37.783798 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:37.783798 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:37.784849 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:37.784849 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:37.784849 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:37.784849 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:37.784849 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:37.784849 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:37.784849 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 12:02:37.784849 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 12:02:37.784849 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 12:02:37.784849 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 12:02:37.784849 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:37.784849 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:37.784849 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:37.784849 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:37.784849 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:37.784849 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:37.785686 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:37.785686 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:37.785686 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:37.785686 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:37.785686 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:37.785686 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:37.785686 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:37.785686 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:37.785686 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:37.785686 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:37.785686 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 12:02:37.785686 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:37.785686 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 12:02:37.785686 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:37.785686 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 476/936] 10.222.0.98 () {56 vars in 1126 bytes} [Thu Aug 18 12:02:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:37.790931 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9debe1-c2f6-4692-9620-30fe4e22b9dc None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:37.814932 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0633119-561f-45e0-822f-8acddd8eeb96 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:37.868476 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:37.903051 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9debe1-c2f6-4692-9620-30fe4e22b9dc None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:37.932459 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0633119-561f-45e0-822f-8acddd8eeb96 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:37.941234 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c767cac-aeeb-4613-8dd3-33559426b1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:37.941865 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c767cac-aeeb-4613-8dd3-33559426b1de None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:37.942149 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c767cac-aeeb-4613-8dd3-33559426b1de None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:37.942429 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c767cac-aeeb-4613-8dd3-33559426b1de None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:37.943077 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c767cac-aeeb-4613-8dd3-33559426b1de None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:37.943386 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c767cac-aeeb-4613-8dd3-33559426b1de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:37.944960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c767cac-aeeb-4613-8dd3-33559426b1de None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:37.981086 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4f7dce-cd01-4667-8f4a-ae5c6568cc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:37.982105 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f7dce-cd01-4667-8f4a-ae5c6568cc42 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:37.982414 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f7dce-cd01-4667-8f4a-ae5c6568cc42 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:37.982692 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f7dce-cd01-4667-8f4a-ae5c6568cc42 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:37.995303 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4f7dce-cd01-4667-8f4a-ae5c6568cc42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:02:38.000410 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-af4f7dce-cd01-4667-8f4a-ae5c6568cc42 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 12:02:38.000410 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:38.000410 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:38.000410 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:38.000410 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:38.000410 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:38.000410 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:38.000410 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:38.000410 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:38.000410 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:38.000410 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:38.000410 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:38.000410 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:02:38.000410 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:02:38.000410 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:02:38.000410 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:02:38.001480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:02:38.001480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:02:38.001480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:02:38.001480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:02:38.001480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:38.001480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:38.001480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:38.001480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:38.001480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:38.001480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:38.001480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:38.001480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:38.001480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:38.001480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:38.001480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:38.001480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:38.002316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:38.002316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:38.002316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:38.002316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:38.002316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:02:38.002316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:02:38.002316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:02:38.002316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:02:38.002316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:02:38.002316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:02:38.002316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 12:02:38.002316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:38.002316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 477/937] 10.222.0.98 () {56 vars in 1120 bytes} [Thu Aug 18 12:02:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:02:38.003131 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4f7dce-cd01-4667-8f4a-ae5c6568cc42 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:38.021839 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 461/938] 10.222.0.98 () {56 vars in 1122 bytes} [Thu Aug 18 12:02:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:38.021839 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c767cac-aeeb-4613-8dd3-33559426b1de None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:38.035330 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:38.120362 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:38.124177 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4f7dce-cd01-4667-8f4a-ae5c6568cc42 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:38.140399 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c767cac-aeeb-4613-8dd3-33559426b1de None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:38.162284 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71446d56-8083-4a91-a226-0a0b3125bcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:38.162925 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71446d56-8083-4a91-a226-0a0b3125bcfc None admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:38.163432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71446d56-8083-4a91-a226-0a0b3125bcfc None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:38.163432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71446d56-8083-4a91-a226-0a0b3125bcfc None admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:38.164004 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71446d56-8083-4a91-a226-0a0b3125bcfc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:38.165436 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71446d56-8083-4a91-a226-0a0b3125bcfc None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:38.168043 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: WARNING keystone.server.flask.common [None req-71446d56-8083-4a91-a226-0a0b3125bcfc None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 18 12:02:38.178783 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf40320-2c0d-45b3-b4b7-83a01e1b336d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:38.179395 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf40320-2c0d-45b3-b4b7-83a01e1b336d None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:38.179711 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf40320-2c0d-45b3-b4b7-83a01e1b336d None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:38.179992 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf40320-2c0d-45b3-b4b7-83a01e1b336d None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:38.180659 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf40320-2c0d-45b3-b4b7-83a01e1b336d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:38.180969 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf40320-2c0d-45b3-b4b7-83a01e1b336d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:38.182322 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf40320-2c0d-45b3-b4b7-83a01e1b336d None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:38.191971 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 462/939] 10.222.0.98 () {56 vars in 1115 bytes} [Thu Aug 18 12:02:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:38.192827 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf40320-2c0d-45b3-b4b7-83a01e1b336d None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:38.224721 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.sql.core [None req-71446d56-8083-4a91-a226-0a0b3125bcfc None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 18 12:02:38.224721 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 18 12:02:38.224721 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [parameters: {'id': '6291dd0a3c1e4b99bf76d0d96d98ef56', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 18 12:02:38.224721 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81863) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 18 12:02:38.231864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-71446d56-8083-4a91-a226-0a0b3125bcfc None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6291dd0a3c1e4b99bf76d0d96d98ef56.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6291dd0a3c1e4b99bf76d0d96d98ef56. Aug 18 12:02:38.231864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:38.231864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 18 12:02:38.231864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 18 12:02:38.231864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 18 12:02:38.231864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 18 12:02:38.231864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 18 12:02:38.231864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application result = self._query(query) Aug 18 12:02:38.231864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 18 12:02:38.231864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application conn.query(q) Aug 18 12:02:38.231864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 18 12:02:38.231864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 18 12:02:38.231864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 18 12:02:38.232693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application result.read() Aug 18 12:02:38.232693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 18 12:02:38.232693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 18 12:02:38.232693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 18 12:02:38.232693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 18 12:02:38.232693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 18 12:02:38.232693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 18 12:02:38.232693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 18 12:02:38.232693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 18 12:02:38.232693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 18 12:02:38.232693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:38.232693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 18 12:02:38.232693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:38.232693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:38.232693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 18 12:02:38.232693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 18 12:02:38.232693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 18 12:02:38.232693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 18 12:02:38.243890 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 18 12:02:38.243890 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application next(self.gen) Aug 18 12:02:38.243890 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 18 12:02:38.243890 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application yield resource Aug 18 12:02:38.243890 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 18 12:02:38.243890 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application next(self.gen) Aug 18 12:02:38.243890 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 18 12:02:38.243890 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self.session.rollback() Aug 18 12:02:38.243890 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 18 12:02:38.243890 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self.force_reraise() Aug 18 12:02:38.243890 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 18 12:02:38.243890 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise self.value Aug 18 12:02:38.243890 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 18 12:02:38.243890 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 18 12:02:38.243890 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 18 12:02:38.243890 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application session.commit() Aug 18 12:02:38.243890 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 18 12:02:38.244797 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 18 12:02:38.244797 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 18 12:02:38.244797 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._prepare_impl() Aug 18 12:02:38.244797 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 18 12:02:38.244797 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self.session.flush() Aug 18 12:02:38.244797 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 18 12:02:38.244797 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._flush(objects) Aug 18 12:02:38.244797 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 18 12:02:38.244797 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 18 12:02:38.244797 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 18 12:02:38.244797 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application compat.raise_( Aug 18 12:02:38.244797 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 18 12:02:38.244797 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception Aug 18 12:02:38.244797 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 18 12:02:38.244797 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application flush_context.execute() Aug 18 12:02:38.244797 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 18 12:02:38.244797 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rec.execute(self) Aug 18 12:02:38.245736 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 18 12:02:38.245736 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 18 12:02:38.245736 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 18 12:02:38.245736 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 18 12:02:38.245736 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 18 12:02:38.245736 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 18 12:02:38.245736 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 18 12:02:38.245736 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 18 12:02:38.245736 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 18 12:02:38.245736 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 18 12:02:38.245736 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 18 12:02:38.245736 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 18 12:02:38.245736 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 18 12:02:38.245736 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 18 12:02:38.245736 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 18 12:02:38.246543 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 18 12:02:38.246543 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 18 12:02:38.246543 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception Aug 18 12:02:38.246543 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 18 12:02:38.246543 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 18 12:02:38.246543 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 18 12:02:38.246543 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 18 12:02:38.246543 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 18 12:02:38.246543 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application result = self._query(query) Aug 18 12:02:38.246543 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 18 12:02:38.246543 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application conn.query(q) Aug 18 12:02:38.246543 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 18 12:02:38.246543 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 18 12:02:38.246543 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 18 12:02:38.246543 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application result.read() Aug 18 12:02:38.246543 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 18 12:02:38.246543 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 18 12:02:38.247583 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 18 12:02:38.247583 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 18 12:02:38.247583 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 18 12:02:38.247583 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 18 12:02:38.247583 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 18 12:02:38.247583 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 18 12:02:38.247583 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 18 12:02:38.247583 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 18 12:02:38.247583 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [parameters: {'id': '6291dd0a3c1e4b99bf76d0d96d98ef56', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 18 12:02:38.247583 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 18 12:02:38.247583 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:38.247583 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 12:02:38.247583 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:38.247583 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:38.247583 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 18 12:02:38.248449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 18 12:02:38.248449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 18 12:02:38.248449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 18 12:02:38.248449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 18 12:02:38.248449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:38.248449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 12:02:38.248449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:38.248449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:38.248449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:38.248449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:38.248449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:38.248449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:38.248449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:38.248449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:38.248449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:38.248449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:38.248449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:38.249553 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:38.249553 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 18 12:02:38.249553 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 18 12:02:38.249553 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:38.249553 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:38.249553 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 18 12:02:38.249553 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 18 12:02:38.249553 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6291dd0a3c1e4b99bf76d0d96d98ef56. Aug 18 12:02:38.249553 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:38.249553 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 478/940] 10.222.0.98 () {60 vars in 1140 bytes} [Thu Aug 18 12:02:38 2022] POST /identity/v3/projects => generated 262 bytes in 231 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:38.249553 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71446d56-8083-4a91-a226-0a0b3125bcfc None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:38.278824 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:38.313365 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf40320-2c0d-45b3-b4b7-83a01e1b336d None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:38.362938 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4fac89-88b0-4eb4-9295-da5b23b3675a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:38.363507 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fac89-88b0-4eb4-9295-da5b23b3675a None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:38.364005 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fac89-88b0-4eb4-9295-da5b23b3675a None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:38.364350 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fac89-88b0-4eb4-9295-da5b23b3675a None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:38.365051 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4fac89-88b0-4eb4-9295-da5b23b3675a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:38.365355 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4fac89-88b0-4eb4-9295-da5b23b3675a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:38.366810 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4fac89-88b0-4eb4-9295-da5b23b3675a None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:38.369414 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71446d56-8083-4a91-a226-0a0b3125bcfc None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:38.372660 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 463/941] 10.222.0.98 () {56 vars in 1122 bytes} [Thu Aug 18 12:02:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:38.372989 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4fac89-88b0-4eb4-9295-da5b23b3675a None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:38.410676 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a287d-d099-4f9e-a10b-5363322d623b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:38.411486 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a287d-d099-4f9e-a10b-5363322d623b None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:38.411883 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a287d-d099-4f9e-a10b-5363322d623b None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:38.412250 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a287d-d099-4f9e-a10b-5363322d623b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:38.423104 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a287d-d099-4f9e-a10b-5363322d623b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:38.425415 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a287d-d099-4f9e-a10b-5363322d623b None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:38.442434 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-c01a287d-d099-4f9e-a10b-5363322d623b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:38.442434 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:38.442434 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:38.442434 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:38.442434 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:38.442434 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:38.442434 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:38.442434 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:38.442434 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:38.442434 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:38.442434 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:38.442434 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:38.442434 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:38.442434 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:38.442434 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:38.442434 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:38.443506 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:38.443506 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:38.443506 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:38.443506 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:38.443506 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:38.443506 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:38.443506 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:38.443506 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:38.443506 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:38.443506 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:38.443506 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:38.443506 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:38.443506 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:38.443506 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:38.443506 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:38.443506 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:38.443506 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:38.444417 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:38.444417 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:38.444417 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:38.444417 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 479/942] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:38.445138 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a287d-d099-4f9e-a10b-5363322d623b None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:38.456731 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:38.495820 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4fac89-88b0-4eb4-9295-da5b23b3675a None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:38.538147 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138613e-d1bf-49a3-acc9-ba43549a8402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:38.538895 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138613e-d1bf-49a3-acc9-ba43549a8402 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:38.539180 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138613e-d1bf-49a3-acc9-ba43549a8402 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:38.539480 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138613e-d1bf-49a3-acc9-ba43549a8402 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:38.542276 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:38.547684 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4138613e-d1bf-49a3-acc9-ba43549a8402 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:38.552117 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4138613e-d1bf-49a3-acc9-ba43549a8402 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:38.553263 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a287d-d099-4f9e-a10b-5363322d623b None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:38.561110 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-4138613e-d1bf-49a3-acc9-ba43549a8402 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:38.561110 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:38.561110 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:38.561110 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:38.561110 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:38.561110 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:38.561110 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:38.561110 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:38.561110 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:38.561110 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:38.561110 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:38.561110 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:38.561110 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 12:02:38.561110 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 12:02:38.561110 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 12:02:38.562016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 12:02:38.562016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:38.562016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:38.562016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:38.562016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:38.562016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:38.562016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:38.562016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:38.562016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:38.562016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:38.562016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:38.562016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:38.562016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:38.562016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:38.562016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:38.562016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:38.562016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:38.564399 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 12:02:38.564399 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 12:02:38.564399 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:02:38.564399 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:02:38.564399 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:02:38.564399 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:02:38.564399 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:38.564399 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:38.564399 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 464/943] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:02:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:38.564399 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138613e-d1bf-49a3-acc9-ba43549a8402 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:38.598747 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925b892-24b8-4095-b174-57dee8fadaff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:38.599678 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925b892-24b8-4095-b174-57dee8fadaff None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:38.599966 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925b892-24b8-4095-b174-57dee8fadaff None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:38.600420 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925b892-24b8-4095-b174-57dee8fadaff None admin] PATH_INFO: `/v3/endpoints` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:38.601444 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3925b892-24b8-4095-b174-57dee8fadaff None admin] RBAC: Adding query filter params (interface=public, service_id=c1445227d84840c9a3e4c0bba2adcbe2, region_id=RegionOne) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:38.601909 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3925b892-24b8-4095-b174-57dee8fadaff None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:38.603291 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3925b892-24b8-4095-b174-57dee8fadaff None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:38.615971 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 480/944] 10.222.0.98 () {56 vars in 1251 bytes} [Thu Aug 18 12:02:38 2022] GET /identity/v3/endpoints?service_id=c1445227d84840c9a3e4c0bba2adcbe2&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:38.615971 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925b892-24b8-4095-b174-57dee8fadaff None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:38.692013 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138613e-d1bf-49a3-acc9-ba43549a8402 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:38.703065 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:38.730863 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f76a50-b942-4879-afbf-5c294ed23d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:38.731505 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f76a50-b942-4879-afbf-5c294ed23d03 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:38.731795 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f76a50-b942-4879-afbf-5c294ed23d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:38.732072 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f76a50-b942-4879-afbf-5c294ed23d03 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:38.732720 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f76a50-b942-4879-afbf-5c294ed23d03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:38.733026 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f76a50-b942-4879-afbf-5c294ed23d03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:38.734397 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f76a50-b942-4879-afbf-5c294ed23d03 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:38.737948 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925b892-24b8-4095-b174-57dee8fadaff None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:38.750933 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 465/945] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:38.750933 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f76a50-b942-4879-afbf-5c294ed23d03 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:38.782353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff19cb4-4592-4eea-8930-823438d3e9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:38.783081 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff19cb4-4592-4eea-8930-823438d3e9e0 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:38.783459 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff19cb4-4592-4eea-8930-823438d3e9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:38.783809 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff19cb4-4592-4eea-8930-823438d3e9e0 None admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:38.784575 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff19cb4-4592-4eea-8930-823438d3e9e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:38.784948 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff19cb4-4592-4eea-8930-823438d3e9e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:38.786448 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff19cb4-4592-4eea-8930-823438d3e9e0 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:38.811444 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 481/946] 10.222.0.98 () {56 vars in 1112 bytes} [Thu Aug 18 12:02:38 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:38.811444 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff19cb4-4592-4eea-8930-823438d3e9e0 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:38.818274 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:38.880951 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f76a50-b942-4879-afbf-5c294ed23d03 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:38.930522 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff19cb4-4592-4eea-8930-823438d3e9e0 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:38.934839 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c28d6f-05d0-47da-acba-a8b0ef28d39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:38.935494 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c28d6f-05d0-47da-acba-a8b0ef28d39d None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:38.935783 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c28d6f-05d0-47da-acba-a8b0ef28d39d None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:38.936062 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c28d6f-05d0-47da-acba-a8b0ef28d39d None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:38.936771 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c28d6f-05d0-47da-acba-a8b0ef28d39d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:38.938167 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c28d6f-05d0-47da-acba-a8b0ef28d39d None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:38.946436 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 466/947] 10.222.0.98 () {56 vars in 1087 bytes} [Thu Aug 18 12:02:38 2022] GET /identity/v3/services => generated 1497 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:02:38.981883 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:38.989234 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c730f7dc-25d8-4ae5-bb7c-47919542f02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:38.990321 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c730f7dc-25d8-4ae5-bb7c-47919542f02a None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:38.990806 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c730f7dc-25d8-4ae5-bb7c-47919542f02a None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:38.991285 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c730f7dc-25d8-4ae5-bb7c-47919542f02a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:39.002945 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c730f7dc-25d8-4ae5-bb7c-47919542f02a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:39.004501 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c730f7dc-25d8-4ae5-bb7c-47919542f02a None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:39.019281 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-c730f7dc-25d8-4ae5-bb7c-47919542f02a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 12:02:39.019281 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:39.019281 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:39.019281 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:39.019281 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:39.019281 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:39.019281 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:39.019281 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:39.019281 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:39.019281 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:39.019281 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:39.019281 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:39.019281 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 12:02:39.019281 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 12:02:39.019281 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 12:02:39.019281 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 12:02:39.020397 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:39.020397 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:39.020397 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:02:39.020397 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:39.020397 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:02:39.020397 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:39.020397 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:39.020397 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:39.020397 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:39.020397 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:39.020397 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:39.020397 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:39.020397 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:39.020397 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:39.020397 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:39.020397 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:39.020397 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:39.020397 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:39.021356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:39.021356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:39.021356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 12:02:39.021356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 12:02:39.021356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 12:02:39.021356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 12:02:39.021356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 12:02:39.021356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 12:02:39.021356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 12:02:39.021356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:39.021356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 482/948] 10.222.0.98 () {56 vars in 1117 bytes} [Thu Aug 18 12:02:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:39.028239 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c28d6f-05d0-47da-acba-a8b0ef28d39d None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:39.156004 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c28d6f-05d0-47da-acba-a8b0ef28d39d None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:39.179225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:39.197961 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9563d33-4f8e-42e2-9c68-2fb6d014a00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:39.197961 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9563d33-4f8e-42e2-9c68-2fb6d014a00f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:39.197961 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9563d33-4f8e-42e2-9c68-2fb6d014a00f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:39.197961 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9563d33-4f8e-42e2-9c68-2fb6d014a00f None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:39.198942 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9563d33-4f8e-42e2-9c68-2fb6d014a00f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:39.198942 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9563d33-4f8e-42e2-9c68-2fb6d014a00f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:39.199990 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9563d33-4f8e-42e2-9c68-2fb6d014a00f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:39.220946 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 467/949] 10.222.0.98 () {56 vars in 1149 bytes} [Thu Aug 18 12:02:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:39.227205 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c730f7dc-25d8-4ae5-bb7c-47919542f02a None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:39.249286 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f3ee8-8b06-4fb2-ab95-ac6f075f4071 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:39.249585 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f3ee8-8b06-4fb2-ab95-ac6f075f4071 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:39.249842 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f3ee8-8b06-4fb2-ab95-ac6f075f4071 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:39.252447 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 468/950] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 12:02:39.256832 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18560b47-22d3-4f77-9bcf-351aaa103efe None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:39.257115 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18560b47-22d3-4f77-9bcf-351aaa103efe None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:39.257728 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18560b47-22d3-4f77-9bcf-351aaa103efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:39.281543 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-18560b47-22d3-4f77-9bcf-351aaa103efe None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:39.282758 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-18560b47-22d3-4f77-9bcf-351aaa103efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:39.338338 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c730f7dc-25d8-4ae5-bb7c-47919542f02a None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:39.366708 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 469/951] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:02:39.379105 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a2a5f3-eb65-4004-adda-f49c0996f06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:39.379773 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a2a5f3-eb65-4004-adda-f49c0996f06e None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:39.380058 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a2a5f3-eb65-4004-adda-f49c0996f06e None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:39.380336 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a2a5f3-eb65-4004-adda-f49c0996f06e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:39.392145 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a2a5f3-eb65-4004-adda-f49c0996f06e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:39.394223 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a2a5f3-eb65-4004-adda-f49c0996f06e None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:39.409634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-50a2a5f3-eb65-4004-adda-f49c0996f06e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:39.409634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:39.409634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:39.409634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:39.409634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:39.409634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:39.409634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:39.409634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:39.409634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:39.409634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:39.409634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:39.409634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:39.409634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:39.409634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:39.409634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:39.409634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:39.410618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:39.410618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:39.410618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:39.410618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:39.410618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:39.410618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:39.410618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:39.410618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:39.410618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:39.410618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:39.410618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:39.410618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:39.410618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:39.410618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:39.410618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:39.410618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:39.410618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:39.411943 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:39.411943 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:39.411943 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:39.411943 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 483/952] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:39.416923 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18560b47-22d3-4f77-9bcf-351aaa103efe None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:39.529476 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18560b47-22d3-4f77-9bcf-351aaa103efe None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:39.541112 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:39.582153 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d41a7-a69c-4d0e-9e39-7d88dc0a7a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:39.583040 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d41a7-a69c-4d0e-9e39-7d88dc0a7a04 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:39.583668 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d41a7-a69c-4d0e-9e39-7d88dc0a7a04 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:39.584355 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d41a7-a69c-4d0e-9e39-7d88dc0a7a04 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:39.585600 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152d41a7-a69c-4d0e-9e39-7d88dc0a7a04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:39.586061 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152d41a7-a69c-4d0e-9e39-7d88dc0a7a04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:39.587637 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152d41a7-a69c-4d0e-9e39-7d88dc0a7a04 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:39.601549 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 470/953] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:39.607397 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a2a5f3-eb65-4004-adda-f49c0996f06e None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:39.636710 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d41a7-a69c-4d0e-9e39-7d88dc0a7a04 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:39.660114 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:39.742170 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a2a5f3-eb65-4004-adda-f49c0996f06e None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:39.772258 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d41a7-a69c-4d0e-9e39-7d88dc0a7a04 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:39.779465 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108a7349-ed49-44cc-82da-cd7cfa39b25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:39.780382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a7349-ed49-44cc-82da-cd7cfa39b25c None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:39.780847 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a7349-ed49-44cc-82da-cd7cfa39b25c None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:39.781301 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a7349-ed49-44cc-82da-cd7cfa39b25c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:39.788944 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108a7349-ed49-44cc-82da-cd7cfa39b25c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:39.790796 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108a7349-ed49-44cc-82da-cd7cfa39b25c None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:39.803576 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-108a7349-ed49-44cc-82da-cd7cfa39b25c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:39.803576 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:39.803576 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:39.803576 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:39.803576 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:39.803576 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:39.803576 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:39.803576 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:39.803576 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:39.803576 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:39.803576 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:39.803576 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:39.803576 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 12:02:39.803576 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 12:02:39.803576 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 12:02:39.804603 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 12:02:39.804603 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:39.804603 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:39.804603 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:39.804603 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:39.804603 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:39.804603 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:39.804603 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:39.804603 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:39.804603 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:39.804603 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:39.804603 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:39.804603 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:39.804603 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:39.804603 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:39.804603 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:39.804603 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:39.805556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 12:02:39.805556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 12:02:39.805556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:02:39.805556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:02:39.805556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:02:39.805556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:02:39.805556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:39.805556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:39.812632 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 484/954] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:02:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:39.813593 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849cc3e1-bf93-43de-88c7-2834178c5112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:39.813958 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108a7349-ed49-44cc-82da-cd7cfa39b25c None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:39.814330 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849cc3e1-bf93-43de-88c7-2834178c5112 None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:39.814699 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849cc3e1-bf93-43de-88c7-2834178c5112 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:39.815056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849cc3e1-bf93-43de-88c7-2834178c5112 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:39.815842 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849cc3e1-bf93-43de-88c7-2834178c5112 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:39.817313 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849cc3e1-bf93-43de-88c7-2834178c5112 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:39.887501 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 471/955] 10.222.0.98 () {60 vars in 1141 bytes} [Thu Aug 18 12:02:39 2022] POST /identity/v3/services => generated 242 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:02:39.937828 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108a7349-ed49-44cc-82da-cd7cfa39b25c None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:39.939728 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:39.986449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd70f59-c7ee-48f2-b8ab-c2de477e248c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:39.987132 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd70f59-c7ee-48f2-b8ab-c2de477e248c None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:39.987522 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd70f59-c7ee-48f2-b8ab-c2de477e248c None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:39.987797 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd70f59-c7ee-48f2-b8ab-c2de477e248c None admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:39.988558 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd70f59-c7ee-48f2-b8ab-c2de477e248c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:39.988869 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd70f59-c7ee-48f2-b8ab-c2de477e248c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:39.990312 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd70f59-c7ee-48f2-b8ab-c2de477e248c None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:40.004442 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 485/956] 10.222.0.98 () {56 vars in 1148 bytes} [Thu Aug 18 12:02:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:40.010430 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849cc3e1-bf93-43de-88c7-2834178c5112 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:40.050532 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:40.137348 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849cc3e1-bf93-43de-88c7-2834178c5112 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:40.176332 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99154d7c-7b50-4862-81a3-136547cb0fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:40.177062 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99154d7c-7b50-4862-81a3-136547cb0fb5 None admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:40.177352 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99154d7c-7b50-4862-81a3-136547cb0fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:40.177632 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99154d7c-7b50-4862-81a3-136547cb0fb5 None admin] PATH_INFO: `/v3/projects/6f676a5169464d83af4267db25025728/users/b6880278f7304051b9ef03e927637e74/roles/9bcab8f185d94480b7fb843825dfb542` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:40.179929 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99154d7c-7b50-4862-81a3-136547cb0fb5 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f676a5169464d83af4267db25025728, user_id=b6880278f7304051b9ef03e927637e74, role_id=9bcab8f185d94480b7fb843825dfb542)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:40.181409 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99154d7c-7b50-4862-81a3-136547cb0fb5 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:40.280152 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 472/957] 10.222.0.98 () {58 vars in 1439 bytes} [Thu Aug 18 12:02:40 2022] PUT /identity/v3/projects/6f676a5169464d83af4267db25025728/users/b6880278f7304051b9ef03e927637e74/roles/9bcab8f185d94480b7fb843825dfb542 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:02:40.941728 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46e130-6538-4eae-8e08-2a2715ec6da4 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:40.942335 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46e130-6538-4eae-8e08-2a2715ec6da4 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:40.942335 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46e130-6538-4eae-8e08-2a2715ec6da4 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:40.943881 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 486/958] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:40.949971 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab4475-f6ba-440e-a01c-dc05a02ec6cd None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:40.950862 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab4475-f6ba-440e-a01c-dc05a02ec6cd None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:40.951327 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab4475-f6ba-440e-a01c-dc05a02ec6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:40.971110 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-fbab4475-f6ba-440e-a01c-dc05a02ec6cd None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:40.972714 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-fbab4475-f6ba-440e-a01c-dc05a02ec6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:41.058580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 473/959] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:40 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:02:41.134037 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619493c-9f97-49c1-b4ad-7a0faf950b6e None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:41.134328 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619493c-9f97-49c1-b4ad-7a0faf950b6e None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:41.134586 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619493c-9f97-49c1-b4ad-7a0faf950b6e None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:41.136250 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 487/960] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:41.145354 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28e6f1-9106-431c-80c1-f0c53552e8b1 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:41.145354 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28e6f1-9106-431c-80c1-f0c53552e8b1 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:41.145354 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28e6f1-9106-431c-80c1-f0c53552e8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:41.171258 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-3d28e6f1-9106-431c-80c1-f0c53552e8b1 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:41.172713 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-3d28e6f1-9106-431c-80c1-f0c53552e8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:41.187675 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:41.291084 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 474/961] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:41 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:02:41.432026 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1619493c-9f97-49c1-b4ad-7a0faf950b6e None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:41.540284 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1619493c-9f97-49c1-b4ad-7a0faf950b6e None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:41.580247 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfea206-75f1-4f23-8a3b-692fcfeb4332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:41.580797 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfea206-75f1-4f23-8a3b-692fcfeb4332 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:41.581077 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfea206-75f1-4f23-8a3b-692fcfeb4332 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:41.581977 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfea206-75f1-4f23-8a3b-692fcfeb4332 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:41.582566 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfea206-75f1-4f23-8a3b-692fcfeb4332 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:41.583961 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfea206-75f1-4f23-8a3b-692fcfeb4332 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:41.592933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-acfea206-75f1-4f23-8a3b-692fcfeb4332 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 12:02:41.592933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:41.592933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 12:02:41.592933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 12:02:41.592933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 12:02:41.592933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 12:02:41.592933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 12:02:41.592933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:41.592933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 12:02:41.592933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:41.592933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 12:02:41.592933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:41.592933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:41.592933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:41.592933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:41.592933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:41.592933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:41.594019 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:41.594019 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:41.594019 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:41.594019 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:41.594019 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:41.594019 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:41.594019 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 12:02:41.594019 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 12:02:41.594019 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 12:02:41.594019 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 12:02:41.594019 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:41.594019 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:41.594019 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:41.594019 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:41.594019 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:41.594019 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:41.594901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:41.594901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:41.594901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:41.594901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:41.594901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:41.594901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:41.594901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:41.594901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:41.594901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:41.594901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:41.594901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 12:02:41.594901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:41.594901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 12:02:41.594901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:41.594901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 488/962] 10.222.0.98 () {56 vars in 1126 bytes} [Thu Aug 18 12:02:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:41.600289 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28e6f1-9106-431c-80c1-f0c53552e8b1 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:41.677064 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfea206-75f1-4f23-8a3b-692fcfeb4332 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:41.705969 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:41.714048 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28e6f1-9106-431c-80c1-f0c53552e8b1 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:41.750035 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f472308-2607-4045-af08-4904a69ff490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:41.750647 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f472308-2607-4045-af08-4904a69ff490 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:41.750976 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f472308-2607-4045-af08-4904a69ff490 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:41.751257 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f472308-2607-4045-af08-4904a69ff490 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:41.751914 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f472308-2607-4045-af08-4904a69ff490 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:41.752262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f472308-2607-4045-af08-4904a69ff490 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:41.754175 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f472308-2607-4045-af08-4904a69ff490 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:41.759299 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 475/963] 10.222.0.98 () {56 vars in 1122 bytes} [Thu Aug 18 12:02:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:41.765199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f472308-2607-4045-af08-4904a69ff490 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:41.781204 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:41.790139 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfea206-75f1-4f23-8a3b-692fcfeb4332 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:41.831122 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee29c6e-2e86-44ef-b027-8db8e58daef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:41.831818 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee29c6e-2e86-44ef-b027-8db8e58daef3 None admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:41.832104 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee29c6e-2e86-44ef-b027-8db8e58daef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:41.834000 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee29c6e-2e86-44ef-b027-8db8e58daef3 None admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:41.834000 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee29c6e-2e86-44ef-b027-8db8e58daef3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:41.834651 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee29c6e-2e86-44ef-b027-8db8e58daef3 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:41.877407 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.sql.core [None req-cee29c6e-2e86-44ef-b027-8db8e58daef3 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 18 12:02:41.877407 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 18 12:02:41.877407 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [parameters: {'user_id': 'ad79018248de4537bf7979a030d6d854', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 18 12:02:41.877407 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81863) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 18 12:02:41.886068 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f472308-2607-4045-af08-4904a69ff490 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:41.887619 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-cee29c6e-2e86-44ef-b027-8db8e58daef3 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 18 12:02:41.887619 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:41.887619 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 18 12:02:41.887619 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 18 12:02:41.887619 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 18 12:02:41.887619 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 18 12:02:41.887619 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 18 12:02:41.887619 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application result = self._query(query) Aug 18 12:02:41.887619 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 18 12:02:41.887619 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application conn.query(q) Aug 18 12:02:41.887619 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 18 12:02:41.887619 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 18 12:02:41.887619 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 18 12:02:41.887619 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application result.read() Aug 18 12:02:41.887619 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 18 12:02:41.888793 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 18 12:02:41.888793 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 18 12:02:41.888793 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 18 12:02:41.888793 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 18 12:02:41.888793 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 18 12:02:41.888793 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 18 12:02:41.888793 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 18 12:02:41.888793 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 18 12:02:41.888793 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:41.888793 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 18 12:02:41.888793 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:41.888793 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:41.888793 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 18 12:02:41.888793 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 18 12:02:41.888793 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 18 12:02:41.888793 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 18 12:02:41.888793 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 18 12:02:41.888793 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application next(self.gen) Aug 18 12:02:41.890310 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 18 12:02:41.890310 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application yield resource Aug 18 12:02:41.890310 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 18 12:02:41.890310 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application next(self.gen) Aug 18 12:02:41.890310 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 18 12:02:41.890310 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self.session.rollback() Aug 18 12:02:41.890310 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 18 12:02:41.890310 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self.force_reraise() Aug 18 12:02:41.890310 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 18 12:02:41.890310 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise self.value Aug 18 12:02:41.890310 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 18 12:02:41.890310 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 18 12:02:41.890310 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 18 12:02:41.890310 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application session.commit() Aug 18 12:02:41.890310 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 18 12:02:41.890310 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 18 12:02:41.890310 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 18 12:02:41.891489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._prepare_impl() Aug 18 12:02:41.891489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 18 12:02:41.891489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self.session.flush() Aug 18 12:02:41.891489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 18 12:02:41.891489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._flush(objects) Aug 18 12:02:41.891489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 18 12:02:41.891489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 18 12:02:41.891489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 18 12:02:41.891489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application compat.raise_( Aug 18 12:02:41.891489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 18 12:02:41.891489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception Aug 18 12:02:41.891489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 18 12:02:41.891489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application flush_context.execute() Aug 18 12:02:41.891489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 18 12:02:41.891489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rec.execute(self) Aug 18 12:02:41.891489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 18 12:02:41.891489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 18 12:02:41.892668 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 18 12:02:41.892668 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 18 12:02:41.892668 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 18 12:02:41.892668 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 18 12:02:41.892668 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 18 12:02:41.892668 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 18 12:02:41.892668 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 18 12:02:41.892668 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 18 12:02:41.892668 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 18 12:02:41.892668 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 18 12:02:41.892668 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 18 12:02:41.892668 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 18 12:02:41.892668 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 18 12:02:41.892668 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 18 12:02:41.892668 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 18 12:02:41.892668 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception Aug 18 12:02:41.893766 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 18 12:02:41.893766 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 18 12:02:41.893766 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 18 12:02:41.893766 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 18 12:02:41.893766 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 18 12:02:41.893766 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application result = self._query(query) Aug 18 12:02:41.893766 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 18 12:02:41.893766 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application conn.query(q) Aug 18 12:02:41.893766 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 18 12:02:41.893766 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 18 12:02:41.893766 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 18 12:02:41.893766 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application result.read() Aug 18 12:02:41.893766 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 18 12:02:41.893766 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 18 12:02:41.893766 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 18 12:02:41.893766 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 18 12:02:41.894972 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 18 12:02:41.894972 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 18 12:02:41.894972 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 18 12:02:41.894972 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 18 12:02:41.894972 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 18 12:02:41.894972 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 18 12:02:41.894972 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ad79018248de4537bf7979a030d6d854', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 18 12:02:41.894972 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 18 12:02:41.894972 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:41.894972 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 12:02:41.894972 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:41.894972 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:41.894972 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:41.894972 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:41.894972 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:41.904495 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:41.904495 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:41.904495 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:41.904495 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:41.904495 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:41.904495 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:41.904495 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:41.904495 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 18 12:02:41.904495 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 18 12:02:41.904495 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:41.904495 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:41.904495 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:02:41.904495 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:41.904495 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:02:41.904495 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:41.904495 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 18 12:02:41.904495 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 18 12:02:41.905775 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 18 12:02:41.905775 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 18 12:02:41.905775 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 18 12:02:41.905775 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 18 12:02:41.905775 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 18 12:02:41.905775 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:41.905775 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 489/964] 10.222.0.98 () {60 vars in 1131 bytes} [Thu Aug 18 12:02:41 2022] POST /identity/v3/users => generated 160 bytes in 213 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:41.905775 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee29c6e-2e86-44ef-b027-8db8e58daef3 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:41.933461 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84afb883-a37f-423c-9e55-33480d613f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:41.934229 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84afb883-a37f-423c-9e55-33480d613f61 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:41.934610 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84afb883-a37f-423c-9e55-33480d613f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:41.935042 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84afb883-a37f-423c-9e55-33480d613f61 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:41.935826 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84afb883-a37f-423c-9e55-33480d613f61 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:41.936226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84afb883-a37f-423c-9e55-33480d613f61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:41.937696 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84afb883-a37f-423c-9e55-33480d613f61 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:41.947562 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 476/965] 10.222.0.98 () {56 vars in 1122 bytes} [Thu Aug 18 12:02:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:41.949550 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84afb883-a37f-423c-9e55-33480d613f61 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:42.009837 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:42.026853 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee29c6e-2e86-44ef-b027-8db8e58daef3 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:42.037440 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:42.062398 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84afb883-a37f-423c-9e55-33480d613f61 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:42.064447 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303f584f-f84f-4fbe-ba1c-62e744b51920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:42.065117 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f584f-f84f-4fbe-ba1c-62e744b51920 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:42.065424 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f584f-f84f-4fbe-ba1c-62e744b51920 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:42.065693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f584f-f84f-4fbe-ba1c-62e744b51920 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:42.073093 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303f584f-f84f-4fbe-ba1c-62e744b51920 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:42.074730 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303f584f-f84f-4fbe-ba1c-62e744b51920 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:42.087293 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-303f584f-f84f-4fbe-ba1c-62e744b51920 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 12:02:42.087293 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:42.087293 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:42.087293 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:42.087293 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:42.087293 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:42.087293 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:42.087293 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:42.087293 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:42.087293 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:42.087293 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:42.087293 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:42.087293 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 12:02:42.087293 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 12:02:42.087293 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 12:02:42.087293 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 12:02:42.088483 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:42.088483 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:42.088483 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:02:42.088483 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:42.088483 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:02:42.088483 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:42.088483 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:42.088483 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:42.088483 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:42.088483 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:42.088483 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:42.088483 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:42.088483 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:42.088483 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:42.088483 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:42.088483 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:42.088483 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:42.088483 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:42.089575 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:42.089575 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:42.089575 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 12:02:42.089575 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 12:02:42.089575 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 12:02:42.089575 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 12:02:42.089575 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 12:02:42.089575 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 12:02:42.089575 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 18 12:02:42.089575 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:42.089575 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 490/966] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:02:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:42.090195 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba9766-aa5f-4e40-9a1a-b0a84f23a534 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:42.090195 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba9766-aa5f-4e40-9a1a-b0a84f23a534 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:42.090195 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba9766-aa5f-4e40-9a1a-b0a84f23a534 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:42.094537 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 491/967] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:42.103273 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba9766-aa5f-4e40-9a1a-b0a84f23a534 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:42.110482 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b033d15e-3cd1-49f2-b54a-9baaaf676ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:42.111490 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033d15e-3cd1-49f2-b54a-9baaaf676ea8 None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:42.111974 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033d15e-3cd1-49f2-b54a-9baaaf676ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:42.112547 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033d15e-3cd1-49f2-b54a-9baaaf676ea8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:42.113310 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b033d15e-3cd1-49f2-b54a-9baaaf676ea8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:42.114752 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b033d15e-3cd1-49f2-b54a-9baaaf676ea8 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:42.210974 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 477/968] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:02:41 2022] POST /identity/v3/endpoints => generated 349 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:02:42.212558 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d918569-345f-4f79-b5d8-d41b48d346c1 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:42.212929 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d918569-345f-4f79-b5d8-d41b48d346c1 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:42.213200 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d918569-345f-4f79-b5d8-d41b48d346c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:42.236850 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-4d918569-345f-4f79-b5d8-d41b48d346c1 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:42.238198 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-4d918569-345f-4f79-b5d8-d41b48d346c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:42.241435 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba9766-aa5f-4e40-9a1a-b0a84f23a534 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:42.284095 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eea7e5-3877-48e8-b09e-594bf9e34d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:42.285759 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eea7e5-3877-48e8-b09e-594bf9e34d2b None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:42.286354 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eea7e5-3877-48e8-b09e-594bf9e34d2b None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:42.286847 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eea7e5-3877-48e8-b09e-594bf9e34d2b None admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:42.287888 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eea7e5-3877-48e8-b09e-594bf9e34d2b None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:42.288462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eea7e5-3877-48e8-b09e-594bf9e34d2b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:42.290210 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eea7e5-3877-48e8-b09e-594bf9e34d2b None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:42.318201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:42.330360 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 478/969] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:42 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:02:42.330360 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef3a57-b117-4132-a7b9-4c897f345b12 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:42.330360 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef3a57-b117-4132-a7b9-4c897f345b12 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:42.330360 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef3a57-b117-4132-a7b9-4c897f345b12 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:42.330360 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 479/970] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:42.338180 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 492/971] 10.222.0.98 () {56 vars in 1101 bytes} [Thu Aug 18 12:02:42 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:42.338284 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d755c-da49-46dc-b392-3c0e676907b2 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:42.338284 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d755c-da49-46dc-b392-3c0e676907b2 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:42.338284 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d755c-da49-46dc-b392-3c0e676907b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:42.360563 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-184d755c-da49-46dc-b392-3c0e676907b2 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:42.362003 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-184d755c-da49-46dc-b392-3c0e676907b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:42.437367 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:42.449421 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 480/972] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:42 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:02:42.617254 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eea7e5-3877-48e8-b09e-594bf9e34d2b None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:42.724668 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d755c-da49-46dc-b392-3c0e676907b2 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:42.728003 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eea7e5-3877-48e8-b09e-594bf9e34d2b None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:42.765432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc254b11-9014-4efc-a9e7-ca9ce5bade75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:42.766065 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc254b11-9014-4efc-a9e7-ca9ce5bade75 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:42.766345 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc254b11-9014-4efc-a9e7-ca9ce5bade75 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:42.766618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc254b11-9014-4efc-a9e7-ca9ce5bade75 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:42.767201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc254b11-9014-4efc-a9e7-ca9ce5bade75 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:42.768632 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc254b11-9014-4efc-a9e7-ca9ce5bade75 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:42.778462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-cc254b11-9014-4efc-a9e7-ca9ce5bade75 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 12:02:42.778462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:42.778462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 12:02:42.778462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 12:02:42.778462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 12:02:42.778462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 12:02:42.778462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 12:02:42.778462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:42.778462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 12:02:42.778462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:42.778462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 12:02:42.778462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:42.778462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:42.778462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:42.778462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:42.778462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:42.778462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:42.779715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:42.779715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:42.779715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:42.779715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:42.779715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:42.779715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:42.779715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 12:02:42.779715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 12:02:42.779715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 12:02:42.779715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 12:02:42.779715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:42.779715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:42.779715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:42.779715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:42.779715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:42.779715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:42.780655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:42.780655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:42.780655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:42.780655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:42.780655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:42.780655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:42.780655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:42.780655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:42.780655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:42.780655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:42.780655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 12:02:42.780655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:42.780655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 12:02:42.780655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:42.780655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 493/973] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:02:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:42.785482 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc254b11-9014-4efc-a9e7-ca9ce5bade75 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:42.838314 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d755c-da49-46dc-b392-3c0e676907b2 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:42.847478 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:42.874850 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4cc56-e8ba-4a2e-88e1-98ddc438dc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:42.875407 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4cc56-e8ba-4a2e-88e1-98ddc438dc2a None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:42.875712 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4cc56-e8ba-4a2e-88e1-98ddc438dc2a None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:42.875993 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4cc56-e8ba-4a2e-88e1-98ddc438dc2a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:42.887079 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c4cc56-e8ba-4a2e-88e1-98ddc438dc2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:02:42.887730 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc254b11-9014-4efc-a9e7-ca9ce5bade75 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:42.892564 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-d6c4cc56-e8ba-4a2e-88e1-98ddc438dc2a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 12:02:42.892564 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:42.892564 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:42.892564 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:42.892564 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:42.892564 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:42.892564 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:42.892564 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:42.892564 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:42.892564 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:42.892564 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:42.892564 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:42.892564 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:02:42.892564 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:02:42.892564 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:02:42.892564 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:02:42.893502 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:02:42.893502 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:02:42.893502 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:02:42.893502 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:02:42.893502 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:42.893502 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:42.893502 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:42.893502 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:42.893502 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:42.893502 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:42.893502 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:42.893502 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:42.893502 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:42.893502 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:42.893502 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:42.893502 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:42.894981 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:42.894981 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:42.894981 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:42.894981 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:42.894981 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:02:42.894981 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:02:42.894981 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:02:42.894981 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:02:42.894981 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:02:42.894981 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:02:42.894981 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 12:02:42.894981 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:42.894981 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 481/974] 10.222.0.98 () {56 vars in 1120 bytes} [Thu Aug 18 12:02:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:42.900151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4cc56-e8ba-4a2e-88e1-98ddc438dc2a None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:42.927707 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c68487-3149-4b40-a708-3e78bc60bd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:42.928325 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c68487-3149-4b40-a708-3e78bc60bd69 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:42.928834 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c68487-3149-4b40-a708-3e78bc60bd69 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:42.928942 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c68487-3149-4b40-a708-3e78bc60bd69 None admin] PATH_INFO: `/v3/services` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:42.929552 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c68487-3149-4b40-a708-3e78bc60bd69 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:42.929850 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c68487-3149-4b40-a708-3e78bc60bd69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:42.931163 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c68487-3149-4b40-a708-3e78bc60bd69 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:42.942186 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 494/975] 10.222.0.98 () {56 vars in 1112 bytes} [Thu Aug 18 12:02:42 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:42.942918 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c68487-3149-4b40-a708-3e78bc60bd69 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:42.958059 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:43.008286 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4cc56-e8ba-4a2e-88e1-98ddc438dc2a None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:43.045732 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648ed555-65a7-497a-8ba7-0beed75294da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:43.046496 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ed555-65a7-497a-8ba7-0beed75294da None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:43.046822 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ed555-65a7-497a-8ba7-0beed75294da None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:43.047098 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ed555-65a7-497a-8ba7-0beed75294da None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:43.047800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648ed555-65a7-497a-8ba7-0beed75294da None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:43.048222 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648ed555-65a7-497a-8ba7-0beed75294da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:43.049621 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648ed555-65a7-497a-8ba7-0beed75294da None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:43.052382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c68487-3149-4b40-a708-3e78bc60bd69 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:43.058664 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 482/976] 10.222.0.98 () {56 vars in 1115 bytes} [Thu Aug 18 12:02:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:02:43.064629 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648ed555-65a7-497a-8ba7-0beed75294da None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:43.093047 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:43.105493 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30757047-7b5e-49fc-904e-3c6b4f200698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:43.106440 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30757047-7b5e-49fc-904e-3c6b4f200698 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:43.106911 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30757047-7b5e-49fc-904e-3c6b4f200698 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:43.109506 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30757047-7b5e-49fc-904e-3c6b4f200698 None admin] PATH_INFO: `/v3/services` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:43.109506 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30757047-7b5e-49fc-904e-3c6b4f200698 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:43.109506 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30757047-7b5e-49fc-904e-3c6b4f200698 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:43.110970 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30757047-7b5e-49fc-904e-3c6b4f200698 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:43.123497 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 495/977] 10.222.0.98 () {56 vars in 1112 bytes} [Thu Aug 18 12:02:42 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:02:43.125761 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30757047-7b5e-49fc-904e-3c6b4f200698 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:43.211156 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648ed555-65a7-497a-8ba7-0beed75294da None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:43.226346 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:43.247813 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c61aa-1fbc-45de-b539-4ed8b5e9e750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:43.248460 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c61aa-1fbc-45de-b539-4ed8b5e9e750 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:43.248748 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c61aa-1fbc-45de-b539-4ed8b5e9e750 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:43.249025 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c61aa-1fbc-45de-b539-4ed8b5e9e750 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:43.251072 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30757047-7b5e-49fc-904e-3c6b4f200698 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:43.261769 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c61aa-1fbc-45de-b539-4ed8b5e9e750 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:43.263240 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c61aa-1fbc-45de-b539-4ed8b5e9e750 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:43.278262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-f97c61aa-1fbc-45de-b539-4ed8b5e9e750 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:43.278262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:43.278262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:43.278262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:43.278262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:43.278262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:43.278262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:43.278262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:43.278262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:43.278262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:43.278262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:43.278262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:43.278262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:43.278262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:43.278262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:43.278262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:43.279273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:43.279273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:43.279273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:43.279273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:43.279273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:43.279273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:43.279273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:43.279273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:43.279273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:43.279273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:43.279273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:43.279273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:43.279273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:43.279273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:43.279273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:43.279273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:43.279273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:43.280709 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:43.280709 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:43.280709 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:43.280709 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 483/978] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:43.285724 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c61aa-1fbc-45de-b539-4ed8b5e9e750 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:43.292228 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65b18f-da74-4655-abff-3e24949e9faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:43.292842 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65b18f-da74-4655-abff-3e24949e9faa None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:43.293126 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65b18f-da74-4655-abff-3e24949e9faa None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:43.293401 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65b18f-da74-4655-abff-3e24949e9faa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:43.294102 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f65b18f-da74-4655-abff-3e24949e9faa None admin] RBAC: Adding query filter params (interface=public, service_id=f77f06577a8946ed8f75c353f91c5432, region_id=RegionOne) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:43.294414 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f65b18f-da74-4655-abff-3e24949e9faa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:43.295964 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f65b18f-da74-4655-abff-3e24949e9faa None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:43.302580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 496/979] 10.222.0.98 () {56 vars in 1251 bytes} [Thu Aug 18 12:02:43 2022] GET /identity/v3/endpoints?service_id=f77f06577a8946ed8f75c353f91c5432&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:02:43.308058 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65b18f-da74-4655-abff-3e24949e9faa None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:43.368846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:43.389980 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c61aa-1fbc-45de-b539-4ed8b5e9e750 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:43.419588 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65b18f-da74-4655-abff-3e24949e9faa None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:43.424177 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c193f-cde5-4407-b907-367d6a255b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:43.424789 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c193f-cde5-4407-b907-367d6a255b4e None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:43.425098 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c193f-cde5-4407-b907-367d6a255b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:43.425407 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c193f-cde5-4407-b907-367d6a255b4e None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:43.426053 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c193f-cde5-4407-b907-367d6a255b4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:43.426366 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c193f-cde5-4407-b907-367d6a255b4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:43.427848 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c193f-cde5-4407-b907-367d6a255b4e None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:43.439955 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 484/980] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:02:43.445444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c193f-cde5-4407-b907-367d6a255b4e None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:43.457717 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b24ab-2582-4f42-9146-315153b61d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:43.458338 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b24ab-2582-4f42-9146-315153b61d24 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:43.458623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b24ab-2582-4f42-9146-315153b61d24 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:43.458903 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b24ab-2582-4f42-9146-315153b61d24 None admin] PATH_INFO: `/v3/services` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:43.459545 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b24ab-2582-4f42-9146-315153b61d24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:43.460916 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b24ab-2582-4f42-9146-315153b61d24 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:43.467122 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 497/981] 10.222.0.98 () {56 vars in 1087 bytes} [Thu Aug 18 12:02:43 2022] GET /identity/v3/services => generated 1727 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 12:02:43.483247 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:43.565086 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c193f-cde5-4407-b907-367d6a255b4e None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:43.602489 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a130410-9ae6-4036-ac7c-bb8d4b11db00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:43.603137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a130410-9ae6-4036-ac7c-bb8d4b11db00 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:43.603436 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a130410-9ae6-4036-ac7c-bb8d4b11db00 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:43.603733 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a130410-9ae6-4036-ac7c-bb8d4b11db00 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:43.611646 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a130410-9ae6-4036-ac7c-bb8d4b11db00 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:43.613645 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a130410-9ae6-4036-ac7c-bb8d4b11db00 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:43.626141 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-5a130410-9ae6-4036-ac7c-bb8d4b11db00 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 12:02:43.626141 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:43.626141 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:43.626141 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:43.626141 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:43.626141 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:43.626141 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:43.626141 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:43.626141 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:43.626141 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:43.626141 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:43.626141 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:43.626141 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 12:02:43.626141 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 12:02:43.626141 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 12:02:43.626141 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 12:02:43.627137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:43.627137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:43.627137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:02:43.627137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:43.627137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:02:43.627137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:43.627137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:43.627137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:43.627137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:43.627137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:43.627137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:43.627137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:43.627137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:43.627137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:43.627137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:43.627137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:43.627137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:43.627137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:43.628103 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:43.628103 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:43.628103 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 12:02:43.628103 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 12:02:43.628103 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 12:02:43.628103 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 12:02:43.628103 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 12:02:43.628103 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 12:02:43.628103 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 18 12:02:43.628103 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:43.630862 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 485/982] 10.222.0.98 () {56 vars in 1117 bytes} [Thu Aug 18 12:02:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:02:43.636382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b24ab-2582-4f42-9146-315153b61d24 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:43.751184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b24ab-2582-4f42-9146-315153b61d24 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:43.770100 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:43.787836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0f7a8-e65f-4fa1-b35f-8f2a82e898e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:43.788501 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0f7a8-e65f-4fa1-b35f-8f2a82e898e3 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:43.788784 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0f7a8-e65f-4fa1-b35f-8f2a82e898e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:43.789058 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0f7a8-e65f-4fa1-b35f-8f2a82e898e3 None admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:43.789838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0f7a8-e65f-4fa1-b35f-8f2a82e898e3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:43.790149 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0f7a8-e65f-4fa1-b35f-8f2a82e898e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:43.791599 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0f7a8-e65f-4fa1-b35f-8f2a82e898e3 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:43.815350 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 498/983] 10.222.0.98 () {56 vars in 1149 bytes} [Thu Aug 18 12:02:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:43.818330 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a130410-9ae6-4036-ac7c-bb8d4b11db00 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:43.907057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:43.922826 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a130410-9ae6-4036-ac7c-bb8d4b11db00 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:43.959398 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f3a8dd-47b7-49d3-8d06-5acd1236ad06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:43.960049 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3a8dd-47b7-49d3-8d06-5acd1236ad06 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:43.960334 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3a8dd-47b7-49d3-8d06-5acd1236ad06 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:43.960611 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3a8dd-47b7-49d3-8d06-5acd1236ad06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:44.032006 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f3a8dd-47b7-49d3-8d06-5acd1236ad06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:44.033371 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f3a8dd-47b7-49d3-8d06-5acd1236ad06 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:44.119595 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:44.140983 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 486/984] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:02:43 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 12:02:44.142186 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0f7a8-e65f-4fa1-b35f-8f2a82e898e3 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:44.242763 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0f7a8-e65f-4fa1-b35f-8f2a82e898e3 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:44.247659 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34fc0f6-32f3-498a-9d64-f77dd6929016 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:44.247947 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34fc0f6-32f3-498a-9d64-f77dd6929016 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:44.248218 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34fc0f6-32f3-498a-9d64-f77dd6929016 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:44.249741 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 487/985] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 12:02:44.255632 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025abc4e-59af-4e26-a02e-6107ffa00f07 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:44.255920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025abc4e-59af-4e26-a02e-6107ffa00f07 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:44.256196 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025abc4e-59af-4e26-a02e-6107ffa00f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:44.274371 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-025abc4e-59af-4e26-a02e-6107ffa00f07 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:44.275612 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-025abc4e-59af-4e26-a02e-6107ffa00f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:44.294818 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a3602-eef9-4a5b-9e1d-25826a14f0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:44.295786 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a3602-eef9-4a5b-9e1d-25826a14f0dc None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:44.296289 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a3602-eef9-4a5b-9e1d-25826a14f0dc None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:44.296763 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a3602-eef9-4a5b-9e1d-25826a14f0dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:44.307935 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539a3602-eef9-4a5b-9e1d-25826a14f0dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:44.309905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539a3602-eef9-4a5b-9e1d-25826a14f0dc None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:44.325577 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-539a3602-eef9-4a5b-9e1d-25826a14f0dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:44.325577 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:44.325577 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:44.325577 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:44.325577 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:44.325577 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:44.325577 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:44.325577 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:44.325577 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:44.325577 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:44.325577 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:44.325577 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:44.325577 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:44.325577 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:44.325577 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:44.325577 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:44.326774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:44.326774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:44.326774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:44.326774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:44.326774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:44.326774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:44.326774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:44.326774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:44.326774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:44.326774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:44.326774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:44.326774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:44.326774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:44.326774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:44.326774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:44.326774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:44.326774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:44.327720 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:44.327720 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:44.327720 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:44.327720 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 499/986] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:44.332822 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a3602-eef9-4a5b-9e1d-25826a14f0dc None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:44.349323 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 488/987] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:44 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 12:02:44.362721 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:44.438703 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a3602-eef9-4a5b-9e1d-25826a14f0dc None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:44.474712 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ecf668-44af-412d-b46e-d7d65645ec3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:44.475331 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ecf668-44af-412d-b46e-d7d65645ec3b None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:44.475645 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ecf668-44af-412d-b46e-d7d65645ec3b None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:44.475924 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ecf668-44af-412d-b46e-d7d65645ec3b None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:44.476623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ecf668-44af-412d-b46e-d7d65645ec3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:44.476928 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ecf668-44af-412d-b46e-d7d65645ec3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:44.478356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ecf668-44af-412d-b46e-d7d65645ec3b None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:44.497997 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 500/988] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:02:44.499312 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025abc4e-59af-4e26-a02e-6107ffa00f07 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:44.606162 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025abc4e-59af-4e26-a02e-6107ffa00f07 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:44.622537 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:44.626259 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ecf668-44af-412d-b46e-d7d65645ec3b None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:44.643131 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d109c8-b360-49d6-90d1-f75e207b5890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:44.643869 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d109c8-b360-49d6-90d1-f75e207b5890 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:44.644722 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d109c8-b360-49d6-90d1-f75e207b5890 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:44.644997 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d109c8-b360-49d6-90d1-f75e207b5890 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:44.652532 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d109c8-b360-49d6-90d1-f75e207b5890 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:44.653982 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d109c8-b360-49d6-90d1-f75e207b5890 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:44.667054 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-65d109c8-b360-49d6-90d1-f75e207b5890 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:44.667054 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:44.667054 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:44.667054 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:44.667054 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:44.667054 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:44.667054 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:44.667054 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:44.667054 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:44.667054 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:44.667054 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:44.667054 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:44.667054 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 12:02:44.667054 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 12:02:44.667054 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 12:02:44.668052 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 12:02:44.668052 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:44.668052 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:44.668052 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:44.668052 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:44.668052 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:44.668052 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:44.668052 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:44.668052 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:44.668052 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:44.668052 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:44.668052 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:44.668052 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:44.668052 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:44.668052 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:44.668052 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:44.668052 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:44.668992 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 12:02:44.668992 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 12:02:44.668992 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:02:44.668992 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:02:44.668992 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:02:44.668992 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:02:44.668992 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:44.668992 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:44.670098 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 489/989] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:02:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:44.674821 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d109c8-b360-49d6-90d1-f75e207b5890 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:44.740485 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:44.752737 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ecf668-44af-412d-b46e-d7d65645ec3b None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:44.788160 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d109c8-b360-49d6-90d1-f75e207b5890 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:44.792732 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe3d1c-df16-489e-84e8-bd6072e36c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:44.793415 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe3d1c-df16-489e-84e8-bd6072e36c22 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:44.793703 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe3d1c-df16-489e-84e8-bd6072e36c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:44.793980 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe3d1c-df16-489e-84e8-bd6072e36c22 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:44.794575 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfe3d1c-df16-489e-84e8-bd6072e36c22 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:44.796002 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfe3d1c-df16-489e-84e8-bd6072e36c22 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:44.806295 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-2bfe3d1c-df16-489e-84e8-bd6072e36c22 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 12:02:44.806295 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:44.806295 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 12:02:44.806295 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 12:02:44.806295 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 12:02:44.806295 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 12:02:44.806295 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 12:02:44.806295 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:44.806295 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 12:02:44.806295 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:44.806295 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 12:02:44.806295 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:44.806295 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:44.806295 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:44.806295 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:44.806295 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:44.806295 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:44.807379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:44.807379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:44.807379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:44.807379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:44.807379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:44.807379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:44.807379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 12:02:44.807379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 12:02:44.807379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 12:02:44.807379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 12:02:44.807379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:44.807379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:44.807379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:44.807379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:44.807379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:44.807379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:44.808304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:44.808304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:44.808304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:44.808304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:44.808304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:44.808304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:44.808304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:44.808304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:44.808304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:44.808304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:44.808304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 12:02:44.808304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:44.808304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 12:02:44.808304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:44.812476 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 501/990] 10.222.0.98 () {56 vars in 1126 bytes} [Thu Aug 18 12:02:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:02:44.813370 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe3d1c-df16-489e-84e8-bd6072e36c22 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:44.830440 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa2e56-84d6-43d6-8e8d-fa0c2240d320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:44.831044 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa2e56-84d6-43d6-8e8d-fa0c2240d320 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:44.831338 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa2e56-84d6-43d6-8e8d-fa0c2240d320 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:44.832232 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa2e56-84d6-43d6-8e8d-fa0c2240d320 None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:44.832936 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa2e56-84d6-43d6-8e8d-fa0c2240d320 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:44.833277 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa2e56-84d6-43d6-8e8d-fa0c2240d320 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:44.834681 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa2e56-84d6-43d6-8e8d-fa0c2240d320 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:44.846822 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 490/991] 10.222.0.98 () {56 vars in 1148 bytes} [Thu Aug 18 12:02:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:44.853858 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa2e56-84d6-43d6-8e8d-fa0c2240d320 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:44.892200 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:44.932966 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe3d1c-df16-489e-84e8-bd6072e36c22 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:44.963799 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa2e56-84d6-43d6-8e8d-fa0c2240d320 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:44.968912 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c057de51-0115-473a-ab98-4081b7da8d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:44.969527 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057de51-0115-473a-ab98-4081b7da8d6c None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:44.969810 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057de51-0115-473a-ab98-4081b7da8d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:44.970120 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057de51-0115-473a-ab98-4081b7da8d6c None admin] PATH_INFO: `/v3/services` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:44.970789 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c057de51-0115-473a-ab98-4081b7da8d6c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:44.971096 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c057de51-0115-473a-ab98-4081b7da8d6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:44.973069 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c057de51-0115-473a-ab98-4081b7da8d6c None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:44.978355 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 502/992] 10.222.0.98 () {56 vars in 1122 bytes} [Thu Aug 18 12:02:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:44.984661 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c057de51-0115-473a-ab98-4081b7da8d6c None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:45.013661 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:45.026766 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acae8eb-adb5-495c-a77c-24d77f2b1714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:45.028158 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acae8eb-adb5-495c-a77c-24d77f2b1714 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:45.028679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acae8eb-adb5-495c-a77c-24d77f2b1714 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:45.029303 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acae8eb-adb5-495c-a77c-24d77f2b1714 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:45.030913 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acae8eb-adb5-495c-a77c-24d77f2b1714 None admin] RBAC: Adding query filter params (role.id=9bcab8f185d94480b7fb843825dfb542, scope.project.id=6f676a5169464d83af4267db25025728, user.id=b6880278f7304051b9ef03e927637e74) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:45.031853 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acae8eb-adb5-495c-a77c-24d77f2b1714 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:45.035530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acae8eb-adb5-495c-a77c-24d77f2b1714 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:45.055459 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 491/993] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:02:44 2022] GET /identity/v3/role_assignments?user.id=b6880278f7304051b9ef03e927637e74&scope.project.id=6f676a5169464d83af4267db25025728&role.id=9bcab8f185d94480b7fb843825dfb542 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:02:45.118638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c057de51-0115-473a-ab98-4081b7da8d6c None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:45.156469 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d30a6-967f-400f-b6c9-8ebc00804015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:45.157155 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d30a6-967f-400f-b6c9-8ebc00804015 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:45.157405 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d30a6-967f-400f-b6c9-8ebc00804015 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:45.157637 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d30a6-967f-400f-b6c9-8ebc00804015 None admin] PATH_INFO: `/v3/services` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:45.158286 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68d30a6-967f-400f-b6c9-8ebc00804015 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:45.158594 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68d30a6-967f-400f-b6c9-8ebc00804015 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:45.160638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68d30a6-967f-400f-b6c9-8ebc00804015 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:45.171498 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 503/994] 10.222.0.98 () {56 vars in 1122 bytes} [Thu Aug 18 12:02:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:02:45.173227 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acae8eb-adb5-495c-a77c-24d77f2b1714 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:45.280470 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:45.283983 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acae8eb-adb5-495c-a77c-24d77f2b1714 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:45.320952 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8b9d44-b764-4fc1-8665-e0fd2340f4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:45.321566 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b9d44-b764-4fc1-8665-e0fd2340f4a6 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:45.321853 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b9d44-b764-4fc1-8665-e0fd2340f4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:45.322132 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b9d44-b764-4fc1-8665-e0fd2340f4a6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:45.322852 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8b9d44-b764-4fc1-8665-e0fd2340f4a6 None admin] RBAC: Adding query filter params (interface=admin, service_id=c1445227d84840c9a3e4c0bba2adcbe2, region_id=RegionOne) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:45.323352 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8b9d44-b764-4fc1-8665-e0fd2340f4a6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:45.325207 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8b9d44-b764-4fc1-8665-e0fd2340f4a6 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:45.335752 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 492/995] 10.222.0.98 () {56 vars in 1249 bytes} [Thu Aug 18 12:02:45 2022] GET /identity/v3/endpoints?service_id=c1445227d84840c9a3e4c0bba2adcbe2&interface=admin®ion_id=RegionOne => generated 193 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:45.337347 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d30a6-967f-400f-b6c9-8ebc00804015 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:45.400879 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded71e2-1ce4-4c56-8574-70128f691794 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:45.401168 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded71e2-1ce4-4c56-8574-70128f691794 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:45.401448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded71e2-1ce4-4c56-8574-70128f691794 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:45.403046 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 493/996] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 12:02:45.408255 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02215448-f22a-4bb5-ad3c-2bc36649974c None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:45.408545 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02215448-f22a-4bb5-ad3c-2bc36649974c None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:45.408825 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02215448-f22a-4bb5-ad3c-2bc36649974c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:45.427120 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-02215448-f22a-4bb5-ad3c-2bc36649974c None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:45.428346 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-02215448-f22a-4bb5-ad3c-2bc36649974c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:45.431238 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:45.456437 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d30a6-967f-400f-b6c9-8ebc00804015 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:45.493638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f9aa6-e865-4b3d-8655-9ebb48194ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:45.494261 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f9aa6-e865-4b3d-8655-9ebb48194ae0 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:45.494549 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f9aa6-e865-4b3d-8655-9ebb48194ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:45.494832 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f9aa6-e865-4b3d-8655-9ebb48194ae0 None admin] PATH_INFO: `/v3/services` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:45.495464 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1f9aa6-e865-4b3d-8655-9ebb48194ae0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:45.496845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1f9aa6-e865-4b3d-8655-9ebb48194ae0 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:45.501376 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 494/997] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:45 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 12:02:45.507401 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 504/998] 10.222.0.98 () {56 vars in 1087 bytes} [Thu Aug 18 12:02:45 2022] GET /identity/v3/services => generated 1727 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:02:45.770094 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02215448-f22a-4bb5-ad3c-2bc36649974c None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:45.867208 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:45.874299 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02215448-f22a-4bb5-ad3c-2bc36649974c None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:45.910269 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e41726-f58c-4ff3-90e1-44b2b95d98aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:45.910919 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e41726-f58c-4ff3-90e1-44b2b95d98aa None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:45.911216 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e41726-f58c-4ff3-90e1-44b2b95d98aa None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:45.911505 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e41726-f58c-4ff3-90e1-44b2b95d98aa None admin] PATH_INFO: `/v3/services/compute` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:45.912123 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e41726-f58c-4ff3-90e1-44b2b95d98aa None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:45.913492 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e41726-f58c-4ff3-90e1-44b2b95d98aa None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:45.922541 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-59e41726-f58c-4ff3-90e1-44b2b95d98aa None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 12:02:45.922541 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:45.922541 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 12:02:45.922541 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 12:02:45.922541 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 12:02:45.922541 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 12:02:45.922541 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 12:02:45.922541 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:45.922541 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 12:02:45.922541 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:45.922541 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 12:02:45.922541 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:45.922541 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:45.922541 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:45.922541 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:45.922541 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:45.922541 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:45.923920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:45.923920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:45.923920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:45.923920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:45.923920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:45.923920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:45.923920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 12:02:45.923920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 12:02:45.923920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 12:02:45.923920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 12:02:45.923920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:45.923920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:45.923920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:45.923920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:45.923920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:45.923920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:45.924761 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:45.924761 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:45.924761 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:45.924761 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:45.924761 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:45.924761 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:45.924761 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:45.924761 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:45.924761 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:45.924761 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:45.924761 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 12:02:45.924761 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:45.924761 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 18 12:02:45.924761 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:45.927493 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 495/999] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:02:45 2022] GET /identity/v3/services/compute => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:45.929719 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f9aa6-e865-4b3d-8655-9ebb48194ae0 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:46.034000 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f9aa6-e865-4b3d-8655-9ebb48194ae0 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:46.069034 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb56422-5e66-413e-9a65-8be4a0d85738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:46.069651 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb56422-5e66-413e-9a65-8be4a0d85738 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:46.069939 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb56422-5e66-413e-9a65-8be4a0d85738 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:46.070249 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb56422-5e66-413e-9a65-8be4a0d85738 None admin] PATH_INFO: `/v3/services` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:46.070854 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb56422-5e66-413e-9a65-8be4a0d85738 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:46.071164 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb56422-5e66-413e-9a65-8be4a0d85738 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:46.072580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb56422-5e66-413e-9a65-8be4a0d85738 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:46.082608 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 505/1000] 10.222.0.98 () {56 vars in 1112 bytes} [Thu Aug 18 12:02:45 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:46.084676 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e41726-f58c-4ff3-90e1-44b2b95d98aa None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:46.189693 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e41726-f58c-4ff3-90e1-44b2b95d98aa None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:46.225007 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3605d-b162-4bcc-851f-daa1b71a908c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:46.225947 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3605d-b162-4bcc-851f-daa1b71a908c None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:46.226459 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3605d-b162-4bcc-851f-daa1b71a908c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:46.226934 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3605d-b162-4bcc-851f-daa1b71a908c None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:46.227789 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed3605d-b162-4bcc-851f-daa1b71a908c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:46.228299 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed3605d-b162-4bcc-851f-daa1b71a908c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:46.229856 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed3605d-b162-4bcc-851f-daa1b71a908c None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:46.239352 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 496/1001] 10.222.0.98 () {56 vars in 1112 bytes} [Thu Aug 18 12:02:46 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:46.241082 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb56422-5e66-413e-9a65-8be4a0d85738 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:46.253319 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:46.346135 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb56422-5e66-413e-9a65-8be4a0d85738 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:46.381440 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11924b-2f9f-4ec5-80d7-d45b327975e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:46.382428 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11924b-2f9f-4ec5-80d7-d45b327975e0 None admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:46.382916 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11924b-2f9f-4ec5-80d7-d45b327975e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:46.383391 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11924b-2f9f-4ec5-80d7-d45b327975e0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:46.384298 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb11924b-2f9f-4ec5-80d7-d45b327975e0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:46.385867 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb11924b-2f9f-4ec5-80d7-d45b327975e0 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:46.507769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 506/1002] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:02:46 2022] POST /identity/v3/endpoints => generated 334 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:02:47.004553 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87daf78-d0d0-40dc-8a3f-02d7dfbf8edd None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:47.004963 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87daf78-d0d0-40dc-8a3f-02d7dfbf8edd None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:47.005160 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87daf78-d0d0-40dc-8a3f-02d7dfbf8edd None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:47.006601 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 497/1003] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 12:02:47.011868 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df951637-9355-4c09-914d-3d437abe0cac None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:47.012322 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df951637-9355-4c09-914d-3d437abe0cac None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:47.012444 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df951637-9355-4c09-914d-3d437abe0cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:47.033047 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-df951637-9355-4c09-914d-3d437abe0cac None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:47.034589 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-df951637-9355-4c09-914d-3d437abe0cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:47.102396 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:47.118440 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 507/1004] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:47 2022] POST /identity/v3/auth/tokens => generated 1726 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:02:47.383984 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87daf78-d0d0-40dc-8a3f-02d7dfbf8edd None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:47.409566 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:47.444067 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab746a-29eb-4a20-95f1-f4b3a1d363a2 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:47.444608 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab746a-29eb-4a20-95f1-f4b3a1d363a2 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:47.444835 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab746a-29eb-4a20-95f1-f4b3a1d363a2 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:47.446937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 508/1005] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 12:02:47.452243 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ce7a7-d6c2-4eb9-8af6-a0f2569f17d3 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:47.452539 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ce7a7-d6c2-4eb9-8af6-a0f2569f17d3 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:47.452826 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ce7a7-d6c2-4eb9-8af6-a0f2569f17d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:47.470721 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-982ce7a7-d6c2-4eb9-8af6-a0f2569f17d3 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:47.471954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-982ce7a7-d6c2-4eb9-8af6-a0f2569f17d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:47.497052 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87daf78-d0d0-40dc-8a3f-02d7dfbf8edd None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:47.536387 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccfff7e-9930-4bc9-9534-2206e205043f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:47.537015 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccfff7e-9930-4bc9-9534-2206e205043f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:47.537294 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccfff7e-9930-4bc9-9534-2206e205043f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:47.537815 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccfff7e-9930-4bc9-9534-2206e205043f None admin] PATH_INFO: `/v3/services/image` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:47.538148 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccfff7e-9930-4bc9-9534-2206e205043f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:47.539541 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccfff7e-9930-4bc9-9534-2206e205043f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:47.545699 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 509/1006] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:47 2022] POST /identity/v3/auth/tokens => generated 1726 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 12:02:47.548546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-6ccfff7e-9930-4bc9-9534-2206e205043f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 12:02:47.548546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:47.548546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 12:02:47.548546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 12:02:47.548546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 12:02:47.548546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 12:02:47.548546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 12:02:47.548546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:47.548546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 12:02:47.548546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:47.548546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 12:02:47.548546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:47.548546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:47.548546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:47.548546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:47.548546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:47.548546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:47.549521 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:47.549521 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:47.549521 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:47.549521 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:47.549521 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:47.549521 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:47.549521 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 12:02:47.549521 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 12:02:47.549521 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 12:02:47.549521 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 12:02:47.549521 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:47.549521 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:47.549521 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:47.549521 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:47.549521 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:47.549521 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:47.550938 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:47.550938 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:47.550938 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:47.550938 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:47.550938 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:47.550938 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:47.550938 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:47.550938 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:47.550938 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:47.550938 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:47.550938 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 12:02:47.550938 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:47.550938 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 12:02:47.550938 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:47.555537 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 498/1007] 10.222.0.98 () {56 vars in 1105 bytes} [Thu Aug 18 12:02:47 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:47.556154 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ce7a7-d6c2-4eb9-8af6-a0f2569f17d3 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:47.658014 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ce7a7-d6c2-4eb9-8af6-a0f2569f17d3 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:47.682412 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:47.702737 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bb8407-275c-45f1-93a7-494922746825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:47.703364 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb8407-275c-45f1-93a7-494922746825 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:47.703669 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb8407-275c-45f1-93a7-494922746825 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:47.703950 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb8407-275c-45f1-93a7-494922746825 None admin] PATH_INFO: `/v3/services` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:47.704593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb8407-275c-45f1-93a7-494922746825 None admin] RBAC: Adding query filter params (name=image) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:47.704903 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb8407-275c-45f1-93a7-494922746825 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:47.706276 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb8407-275c-45f1-93a7-494922746825 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:47.715934 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 510/1008] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:47 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:02:47.716940 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccfff7e-9930-4bc9-9534-2206e205043f None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:47.807063 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:47.816435 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bb8407-275c-45f1-93a7-494922746825 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:47.823795 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccfff7e-9930-4bc9-9534-2206e205043f None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:47.861340 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc8642d-f24f-43b5-9c19-a73e6eea37d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:47.861950 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8642d-f24f-43b5-9c19-a73e6eea37d9 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:47.862235 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8642d-f24f-43b5-9c19-a73e6eea37d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:47.862515 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8642d-f24f-43b5-9c19-a73e6eea37d9 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:47.863741 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc8642d-f24f-43b5-9c19-a73e6eea37d9 None admin] RBAC: Adding query filter params (type=image) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:47.864079 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc8642d-f24f-43b5-9c19-a73e6eea37d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:47.865452 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc8642d-f24f-43b5-9c19-a73e6eea37d9 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:47.879457 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 499/1009] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:47 2022] GET /identity/v3/services?type=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:47.934626 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bb8407-275c-45f1-93a7-494922746825 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:47.973441 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c18098-e2f9-4713-812e-21e1a337beb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:47.974095 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c18098-e2f9-4713-812e-21e1a337beb1 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:47.974398 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c18098-e2f9-4713-812e-21e1a337beb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:47.974647 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c18098-e2f9-4713-812e-21e1a337beb1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:47.975253 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c18098-e2f9-4713-812e-21e1a337beb1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:47.976705 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c18098-e2f9-4713-812e-21e1a337beb1 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:47.986951 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-54c18098-e2f9-4713-812e-21e1a337beb1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 12:02:47.986951 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:47.986951 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 12:02:47.986951 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 12:02:47.986951 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 12:02:47.986951 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 12:02:47.986951 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 12:02:47.986951 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:47.986951 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 12:02:47.986951 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:47.986951 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 12:02:47.986951 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:47.986951 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:47.986951 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:47.986951 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:47.986951 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:47.986951 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:47.988094 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:47.988094 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:47.988094 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:47.988094 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:47.988094 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:47.988094 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:47.988094 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 12:02:47.988094 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 12:02:47.988094 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 12:02:47.988094 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 12:02:47.988094 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:47.988094 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:47.988094 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:47.988094 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:47.988094 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:47.988094 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:47.992525 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:47.992525 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:47.992525 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:47.992525 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:47.992525 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:47.992525 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:47.992525 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:47.992525 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:47.992525 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:47.992525 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:47.992525 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 12:02:47.992525 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:47.992525 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 18 12:02:47.992525 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:47.992525 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 511/1010] 10.222.0.98 () {56 vars in 1126 bytes} [Thu Aug 18 12:02:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:02:47.994882 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc8642d-f24f-43b5-9c19-a73e6eea37d9 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:48.105467 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc8642d-f24f-43b5-9c19-a73e6eea37d9 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:48.140239 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd695f-8e7b-437e-85d7-37b00e60eb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:48.140859 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd695f-8e7b-437e-85d7-37b00e60eb3e None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:48.141143 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd695f-8e7b-437e-85d7-37b00e60eb3e None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:48.141422 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd695f-8e7b-437e-85d7-37b00e60eb3e None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:48.142057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd695f-8e7b-437e-85d7-37b00e60eb3e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:48.142468 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd695f-8e7b-437e-85d7-37b00e60eb3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:48.143767 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd695f-8e7b-437e-85d7-37b00e60eb3e None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:48.149710 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 500/1011] 10.222.0.98 () {56 vars in 1122 bytes} [Thu Aug 18 12:02:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:48.155557 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c18098-e2f9-4713-812e-21e1a337beb1 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:48.207764 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:48.260625 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c18098-e2f9-4713-812e-21e1a337beb1 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:48.296288 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c68f996-73ca-4048-928c-ae381eb408db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:48.296903 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68f996-73ca-4048-928c-ae381eb408db None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:48.297186 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68f996-73ca-4048-928c-ae381eb408db None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:48.297456 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68f996-73ca-4048-928c-ae381eb408db None admin] PATH_INFO: `/v3/services` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:48.298092 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c68f996-73ca-4048-928c-ae381eb408db None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:48.298400 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c68f996-73ca-4048-928c-ae381eb408db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:48.299839 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c68f996-73ca-4048-928c-ae381eb408db None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:48.305487 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 512/1012] 10.222.0.98 () {56 vars in 1122 bytes} [Thu Aug 18 12:02:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:48.311016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd695f-8e7b-437e-85d7-37b00e60eb3e None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:48.330577 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:48.420960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd695f-8e7b-437e-85d7-37b00e60eb3e None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:48.458517 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf532f-694e-4443-88a7-f7ec4291a35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:48.459079 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf532f-694e-4443-88a7-f7ec4291a35d None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:48.459362 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf532f-694e-4443-88a7-f7ec4291a35d None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:48.459665 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf532f-694e-4443-88a7-f7ec4291a35d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:48.460353 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acf532f-694e-4443-88a7-f7ec4291a35d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:48.461720 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acf532f-694e-4443-88a7-f7ec4291a35d None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:48.537395 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a7daf-8743-4afb-b59f-af9082fe30e2 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:48.537976 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a7daf-8743-4afb-b59f-af9082fe30e2 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:48.537976 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a7daf-8743-4afb-b59f-af9082fe30e2 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:48.539310 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 513/1013] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 12:02:48.545125 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3e1e4-0ee9-490f-8b46-7ec7860548fa None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:48.545513 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3e1e4-0ee9-490f-8b46-7ec7860548fa None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:48.545873 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3e1e4-0ee9-490f-8b46-7ec7860548fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:48.549277 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 501/1014] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:02:48 2022] POST /identity/v3/endpoints => generated 325 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:02:48.569687 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-63b3e1e4-0ee9-490f-8b46-7ec7860548fa None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:48.571005 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-63b3e1e4-0ee9-490f-8b46-7ec7860548fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:48.664546 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 514/1015] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:48 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 12:02:48.938972 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf532f-694e-4443-88a7-f7ec4291a35d None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:49.021678 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:49.051392 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf532f-694e-4443-88a7-f7ec4291a35d None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:49.086290 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03fa9e-3a87-4b41-9af4-66eef1867398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:49.086951 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03fa9e-3a87-4b41-9af4-66eef1867398 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:49.087282 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03fa9e-3a87-4b41-9af4-66eef1867398 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:49.087590 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03fa9e-3a87-4b41-9af4-66eef1867398 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:49.097882 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b03fa9e-3a87-4b41-9af4-66eef1867398 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:02:49.103139 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-7b03fa9e-3a87-4b41-9af4-66eef1867398 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 12:02:49.103139 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:49.103139 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:49.103139 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:49.103139 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:49.103139 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:49.103139 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:49.103139 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:49.103139 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:49.103139 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:49.103139 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:49.103139 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:49.103139 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:02:49.103139 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:02:49.103139 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:02:49.103139 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:02:49.104097 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:02:49.104097 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:02:49.104097 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:02:49.104097 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:02:49.104097 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:49.104097 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:49.104097 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:49.104097 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:49.104097 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:49.104097 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:49.104097 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:49.104097 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:49.104097 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:49.104097 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:49.104097 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:49.104097 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:49.104920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:49.104920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:49.104920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:49.104920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:49.104920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:02:49.104920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:02:49.104920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:02:49.104920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:02:49.104920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:02:49.104920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:02:49.104920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 12:02:49.104920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:49.104920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 502/1016] 10.222.0.98 () {56 vars in 1120 bytes} [Thu Aug 18 12:02:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:49.111992 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3e1e4-0ee9-490f-8b46-7ec7860548fa None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:49.153074 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:49.213523 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3e1e4-0ee9-490f-8b46-7ec7860548fa None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:49.248862 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f59a4-2804-47d2-929a-769176ce638b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:49.249468 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f59a4-2804-47d2-929a-769176ce638b None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:49.249752 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f59a4-2804-47d2-929a-769176ce638b None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:49.250027 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f59a4-2804-47d2-929a-769176ce638b None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:49.250686 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6f59a4-2804-47d2-929a-769176ce638b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:49.250993 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6f59a4-2804-47d2-929a-769176ce638b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:49.252377 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6f59a4-2804-47d2-929a-769176ce638b None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:49.265058 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 515/1017] 10.222.0.98 () {56 vars in 1115 bytes} [Thu Aug 18 12:02:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:49.266656 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03fa9e-3a87-4b41-9af4-66eef1867398 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:49.380780 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03fa9e-3a87-4b41-9af4-66eef1867398 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:49.415727 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d0fb1-6f52-4e61-acb7-1a077083d82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:49.416355 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d0fb1-6f52-4e61-acb7-1a077083d82b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:49.416635 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d0fb1-6f52-4e61-acb7-1a077083d82b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:49.416908 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d0fb1-6f52-4e61-acb7-1a077083d82b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:49.418948 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:49.428899 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992d0fb1-6f52-4e61-acb7-1a077083d82b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:49.430347 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992d0fb1-6f52-4e61-acb7-1a077083d82b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:49.444203 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-992d0fb1-6f52-4e61-acb7-1a077083d82b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:49.444203 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:49.444203 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:49.444203 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:49.444203 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:49.444203 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:49.444203 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:49.444203 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:49.444203 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:49.444203 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:49.444203 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:49.444203 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:49.444203 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:49.444203 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:49.444203 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:49.444203 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:49.445169 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:49.445169 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:49.445169 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:49.445169 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:49.445169 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:49.445169 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:49.445169 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:49.445169 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:49.445169 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:49.445169 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:49.445169 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:49.445169 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:49.445169 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:49.445169 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:49.445169 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:49.445169 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:49.445169 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:49.446074 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:49.446074 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:49.446074 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:49.446074 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 503/1018] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:02:49.451361 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f59a4-2804-47d2-929a-769176ce638b None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:49.557316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:49.561582 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f59a4-2804-47d2-929a-769176ce638b None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:49.595648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c9326-881a-4a2b-9287-4daeec72fd61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:49.596261 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c9326-881a-4a2b-9287-4daeec72fd61 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:49.596547 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c9326-881a-4a2b-9287-4daeec72fd61 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:49.596831 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c9326-881a-4a2b-9287-4daeec72fd61 None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:49.598384 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6c9326-881a-4a2b-9287-4daeec72fd61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:49.598864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6c9326-881a-4a2b-9287-4daeec72fd61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:49.600663 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6c9326-881a-4a2b-9287-4daeec72fd61 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:49.612801 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 516/1019] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:49.618431 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d0fb1-6f52-4e61-acb7-1a077083d82b None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:49.747297 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d0fb1-6f52-4e61-acb7-1a077083d82b None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:49.781675 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343dc412-92a8-4010-b6af-651188766a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:49.782301 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343dc412-92a8-4010-b6af-651188766a2a None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:49.782578 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343dc412-92a8-4010-b6af-651188766a2a None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:49.782846 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343dc412-92a8-4010-b6af-651188766a2a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:49.789695 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a76f21-a972-4605-aeb8-2ab88d120a99 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:49.789986 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a76f21-a972-4605-aeb8-2ab88d120a99 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:49.790280 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a76f21-a972-4605-aeb8-2ab88d120a99 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:49.791367 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343dc412-92a8-4010-b6af-651188766a2a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:49.791651 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 517/1020] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 12:02:49.793158 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343dc412-92a8-4010-b6af-651188766a2a None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:49.796495 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25423f-201d-46f7-89ff-d5c7fe693d84 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:49.796786 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25423f-201d-46f7-89ff-d5c7fe693d84 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:49.797065 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25423f-201d-46f7-89ff-d5c7fe693d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:49.806405 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-343dc412-92a8-4010-b6af-651188766a2a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 12:02:49.806405 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:49.806405 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:49.806405 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:49.806405 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:49.806405 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:49.806405 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:49.806405 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:49.806405 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:49.806405 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:49.806405 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:49.806405 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:49.806405 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 12:02:49.806405 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 12:02:49.806405 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 12:02:49.806405 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 12:02:49.807615 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:49.807615 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:49.807615 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:02:49.807615 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:49.807615 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:02:49.807615 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:49.807615 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:49.807615 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:49.807615 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:49.807615 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:49.807615 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:49.807615 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:49.807615 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:49.807615 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:49.807615 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:49.807615 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:49.807615 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:49.807615 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:49.808576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:49.808576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:49.808576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 12:02:49.808576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 12:02:49.808576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 12:02:49.808576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 12:02:49.808576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 12:02:49.808576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 12:02:49.808576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 12:02:49.808576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:49.811672 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 504/1021] 10.222.0.98 () {56 vars in 1093 bytes} [Thu Aug 18 12:02:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:02:49.814569 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343dc412-92a8-4010-b6af-651188766a2a None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:49.815349 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-4f25423f-201d-46f7-89ff-d5c7fe693d84 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:49.817124 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-4f25423f-201d-46f7-89ff-d5c7fe693d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:49.862956 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:49.890246 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 518/1022] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:49 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 12:02:49.917867 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343dc412-92a8-4010-b6af-651188766a2a None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:49.952693 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01850f-7f1c-48a9-8582-d5de2eba6e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:49.957669 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01850f-7f1c-48a9-8582-d5de2eba6e71 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:49.957669 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01850f-7f1c-48a9-8582-d5de2eba6e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:49.957669 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01850f-7f1c-48a9-8582-d5de2eba6e71 None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:49.957669 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac01850f-7f1c-48a9-8582-d5de2eba6e71 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:49.957669 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac01850f-7f1c-48a9-8582-d5de2eba6e71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:49.957669 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac01850f-7f1c-48a9-8582-d5de2eba6e71 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:49.975108 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 505/1023] 10.222.0.98 () {56 vars in 1133 bytes} [Thu Aug 18 12:02:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:02:49.980672 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f25423f-201d-46f7-89ff-d5c7fe693d84 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:50.073377 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:50.083758 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f25423f-201d-46f7-89ff-d5c7fe693d84 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:50.117728 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831b55c-2a0d-4dd8-abb4-15d5e7b76aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:50.118571 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831b55c-2a0d-4dd8-abb4-15d5e7b76aee None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:50.118893 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831b55c-2a0d-4dd8-abb4-15d5e7b76aee None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:50.119342 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831b55c-2a0d-4dd8-abb4-15d5e7b76aee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:50.159738 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01850f-7f1c-48a9-8582-d5de2eba6e71 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:50.190931 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9831b55c-2a0d-4dd8-abb4-15d5e7b76aee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:50.193029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9831b55c-2a0d-4dd8-abb4-15d5e7b76aee None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:50.264483 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01850f-7f1c-48a9-8582-d5de2eba6e71 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:50.267609 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:50.292369 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:50.303298 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 519/1024] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:02:49 2022] GET /identity/v3/auth/tokens => generated 1873 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 12:02:50.308697 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831b55c-2a0d-4dd8-abb4-15d5e7b76aee None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:50.309841 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd026b-88c3-491a-bbba-46cf6b06b322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:50.310471 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd026b-88c3-491a-bbba-46cf6b06b322 None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:50.310755 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd026b-88c3-491a-bbba-46cf6b06b322 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:50.311050 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd026b-88c3-491a-bbba-46cf6b06b322 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:50.311876 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd026b-88c3-491a-bbba-46cf6b06b322 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:50.313256 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd026b-88c3-491a-bbba-46cf6b06b322 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:50.409601 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831b55c-2a0d-4dd8-abb4-15d5e7b76aee None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:50.429646 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 506/1025] 10.222.0.98 () {60 vars in 1141 bytes} [Thu Aug 18 12:02:50 2022] POST /identity/v3/services => generated 242 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:02:50.466428 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b7d15f-2be5-4fa0-a8e6-ee3cd85ae56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:50.467142 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b7d15f-2be5-4fa0-a8e6-ee3cd85ae56c None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:50.467461 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b7d15f-2be5-4fa0-a8e6-ee3cd85ae56c None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:50.467735 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b7d15f-2be5-4fa0-a8e6-ee3cd85ae56c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:50.479848 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b7d15f-2be5-4fa0-a8e6-ee3cd85ae56c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:50.481351 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b7d15f-2be5-4fa0-a8e6-ee3cd85ae56c None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:50.499389 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-72b7d15f-2be5-4fa0-a8e6-ee3cd85ae56c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:50.499389 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:50.499389 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:50.499389 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:50.499389 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:50.499389 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:50.499389 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:50.499389 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:50.499389 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:50.499389 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:50.499389 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:50.499389 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:50.499389 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:50.499389 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:50.499389 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:50.499389 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:50.500519 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:50.500519 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:50.500519 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:50.500519 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:50.500519 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:50.500519 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:50.500519 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:50.500519 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:50.500519 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:50.500519 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:50.500519 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:50.500519 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:50.500519 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:50.500519 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:50.500519 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:50.500519 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:50.500519 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:50.507088 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:50.507088 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:50.507088 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:50.507088 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 520/1026] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:02:50.508884 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd026b-88c3-491a-bbba-46cf6b06b322 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:50.555752 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba4f55-3b8f-4d33-908e-a6e3e7ea7a5a None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:50.556129 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba4f55-3b8f-4d33-908e-a6e3e7ea7a5a None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:50.556301 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba4f55-3b8f-4d33-908e-a6e3e7ea7a5a None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:50.558232 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 521/1027] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 12:02:50.563950 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcdf791-e68e-46fa-8103-b5535ddec2a3 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:50.564235 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcdf791-e68e-46fa-8103-b5535ddec2a3 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:50.564512 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcdf791-e68e-46fa-8103-b5535ddec2a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:50.584294 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-2fcdf791-e68e-46fa-8103-b5535ddec2a3 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:50.585501 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-2fcdf791-e68e-46fa-8103-b5535ddec2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:50.620341 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:50.640358 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd026b-88c3-491a-bbba-46cf6b06b322 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:50.665030 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 522/1028] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:50 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 12:02:50.680347 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926be77-2f5b-4c92-820f-01c66777dee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:50.680970 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926be77-2f5b-4c92-820f-01c66777dee0 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:50.681253 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926be77-2f5b-4c92-820f-01c66777dee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:50.681533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926be77-2f5b-4c92-820f-01c66777dee0 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:50.682195 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3926be77-2f5b-4c92-820f-01c66777dee0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:50.682504 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3926be77-2f5b-4c92-820f-01c66777dee0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:50.683923 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3926be77-2f5b-4c92-820f-01c66777dee0 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:50.703454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 507/1029] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:50.705423 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcdf791-e68e-46fa-8103-b5535ddec2a3 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:50.818320 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcdf791-e68e-46fa-8103-b5535ddec2a3 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:50.853413 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee9073-6aea-4629-bb24-56007f3e405d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:50.854050 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee9073-6aea-4629-bb24-56007f3e405d None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:50.854335 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee9073-6aea-4629-bb24-56007f3e405d None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:50.854614 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee9073-6aea-4629-bb24-56007f3e405d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:50.861497 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee9073-6aea-4629-bb24-56007f3e405d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:50.862956 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee9073-6aea-4629-bb24-56007f3e405d None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:50.874838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-e1ee9073-6aea-4629-bb24-56007f3e405d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:50.874838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:50.874838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:50.874838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:50.874838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:50.874838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:50.874838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:50.874838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:50.874838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:50.874838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:50.874838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:50.874838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:50.874838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 12:02:50.874838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 12:02:50.874838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 12:02:50.876029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 12:02:50.876029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:50.876029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:50.876029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:50.876029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:50.876029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:50.876029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:50.876029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:50.876029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:50.876029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:50.876029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:50.876029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:50.876029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:50.876029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:50.876029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:50.876029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:50.876029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:50.876928 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 12:02:50.876928 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 12:02:50.876928 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:02:50.876928 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:02:50.876928 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:02:50.876928 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:02:50.876928 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:50.876928 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:50.876928 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 523/1030] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:02:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:50.883620 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926be77-2f5b-4c92-820f-01c66777dee0 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:50.915883 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:50.940785 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee9073-6aea-4629-bb24-56007f3e405d None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:50.975625 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:50.995611 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926be77-2f5b-4c92-820f-01c66777dee0 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:51.040754 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c7e01-f6c1-476e-8b90-e82b20176406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:51.041790 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c7e01-f6c1-476e-8b90-e82b20176406 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:51.042347 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c7e01-f6c1-476e-8b90-e82b20176406 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:51.042830 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c7e01-f6c1-476e-8b90-e82b20176406 None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:51.044095 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c7e01-f6c1-476e-8b90-e82b20176406 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:51.044363 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c7e01-f6c1-476e-8b90-e82b20176406 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:51.045849 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c7e01-f6c1-476e-8b90-e82b20176406 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:51.064332 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 508/1031] 10.222.0.98 () {56 vars in 1148 bytes} [Thu Aug 18 12:02:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:51.064526 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee9073-6aea-4629-bb24-56007f3e405d None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:51.065425 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c7e01-f6c1-476e-8b90-e82b20176406 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:51.102007 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f2b86c-973b-4d3d-8a62-04d5b2451be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:51.102227 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f2b86c-973b-4d3d-8a62-04d5b2451be4 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:51.102509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f2b86c-973b-4d3d-8a62-04d5b2451be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:51.102774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f2b86c-973b-4d3d-8a62-04d5b2451be4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:51.104044 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f2b86c-973b-4d3d-8a62-04d5b2451be4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:51.105363 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f2b86c-973b-4d3d-8a62-04d5b2451be4 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:51.107154 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 524/1032] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:02:51.113735 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f2b86c-973b-4d3d-8a62-04d5b2451be4 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:51.157911 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:51.175003 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c7e01-f6c1-476e-8b90-e82b20176406 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:51.213798 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d807063-3bee-4d52-a9e4-a9ce57023ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:51.214471 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d807063-3bee-4d52-a9e4-a9ce57023ec5 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:51.214756 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d807063-3bee-4d52-a9e4-a9ce57023ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:51.215030 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d807063-3bee-4d52-a9e4-a9ce57023ec5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:51.215793 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d807063-3bee-4d52-a9e4-a9ce57023ec5 None admin] RBAC: Adding query filter params (role.id=9bcab8f185d94480b7fb843825dfb542, scope.project.id=6f676a5169464d83af4267db25025728, user.id=171c72a6b29e418bbb4ea5c6faffe3b3) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:51.216131 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d807063-3bee-4d52-a9e4-a9ce57023ec5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:51.217511 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d807063-3bee-4d52-a9e4-a9ce57023ec5 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:51.222075 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:51.229279 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f2b86c-973b-4d3d-8a62-04d5b2451be4 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:51.231919 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 509/1033] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:02:51 2022] GET /identity/v3/role_assignments?user.id=171c72a6b29e418bbb4ea5c6faffe3b3&scope.project.id=6f676a5169464d83af4267db25025728&role.id=9bcab8f185d94480b7fb843825dfb542 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:02:51.268979 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1bdf8-8969-4118-9634-7f8a243f12a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:51.270262 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1bdf8-8969-4118-9634-7f8a243f12a7 None admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:51.270521 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1bdf8-8969-4118-9634-7f8a243f12a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:51.270802 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1bdf8-8969-4118-9634-7f8a243f12a7 None admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:51.271504 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1bdf8-8969-4118-9634-7f8a243f12a7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:51.272994 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1bdf8-8969-4118-9634-7f8a243f12a7 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:51.361965 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 525/1034] 10.222.0.98 () {60 vars in 1141 bytes} [Thu Aug 18 12:02:51 2022] POST /identity/v3/projects => generated 311 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:02:52.404845 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745f60e-1fc4-415c-bcaa-b785b4354659 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:52.405376 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745f60e-1fc4-415c-bcaa-b785b4354659 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:52.405376 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745f60e-1fc4-415c-bcaa-b785b4354659 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:52.406856 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 510/1035] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:52.411840 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6de2d-130f-4eb4-bd6c-53c6dd7fdf43 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:52.412261 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6de2d-130f-4eb4-bd6c-53c6dd7fdf43 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:52.412433 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6de2d-130f-4eb4-bd6c-53c6dd7fdf43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:52.430918 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-81e6de2d-130f-4eb4-bd6c-53c6dd7fdf43 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:52.432669 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-81e6de2d-130f-4eb4-bd6c-53c6dd7fdf43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:52.520648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 526/1036] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:52 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:02:52.790353 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745f60e-1fc4-415c-bcaa-b785b4354659 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:52.892984 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745f60e-1fc4-415c-bcaa-b785b4354659 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:52.928521 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3fcaca-6fe6-439c-bc6d-5cc95ceab7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:52.929148 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3fcaca-6fe6-439c-bc6d-5cc95ceab7e5 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:52.930610 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3fcaca-6fe6-439c-bc6d-5cc95ceab7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:52.930610 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3fcaca-6fe6-439c-bc6d-5cc95ceab7e5 None admin] PATH_INFO: `/v3/services/image` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:52.930610 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3fcaca-6fe6-439c-bc6d-5cc95ceab7e5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:52.931851 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3fcaca-6fe6-439c-bc6d-5cc95ceab7e5 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:52.940841 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-fa3fcaca-6fe6-439c-bc6d-5cc95ceab7e5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 12:02:52.940841 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:52.940841 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 12:02:52.940841 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 12:02:52.940841 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 12:02:52.940841 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 12:02:52.940841 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 12:02:52.940841 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:52.940841 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 12:02:52.940841 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:52.940841 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 12:02:52.940841 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:52.940841 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:52.940841 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:52.940841 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:52.940841 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:52.940841 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:52.941869 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:52.941869 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:52.941869 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:52.941869 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:52.941869 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:52.941869 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:52.941869 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 12:02:52.941869 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 12:02:52.941869 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 12:02:52.941869 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 12:02:52.941869 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:52.941869 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:52.941869 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:52.941869 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:52.941869 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:52.941869 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:52.942739 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:52.942739 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:52.942739 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:52.942739 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:52.942739 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:52.942739 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:52.942739 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:52.942739 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:52.942739 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:52.942739 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:52.942739 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 12:02:52.942739 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:52.942739 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 12:02:52.942739 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:52.944453 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 511/1037] 10.222.0.98 () {56 vars in 1105 bytes} [Thu Aug 18 12:02:52 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:52.948109 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e6de2d-130f-4eb4-bd6c-53c6dd7fdf43 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:53.027151 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:53.049411 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e6de2d-130f-4eb4-bd6c-53c6dd7fdf43 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:53.084987 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720540aa-633a-4631-b2eb-f3053337efd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:53.085843 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720540aa-633a-4631-b2eb-f3053337efd8 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:53.086275 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720540aa-633a-4631-b2eb-f3053337efd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:53.086700 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720540aa-633a-4631-b2eb-f3053337efd8 None admin] PATH_INFO: `/v3/services` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:53.089866 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720540aa-633a-4631-b2eb-f3053337efd8 None admin] RBAC: Adding query filter params (name=image) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:53.089866 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720540aa-633a-4631-b2eb-f3053337efd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:53.089866 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720540aa-633a-4631-b2eb-f3053337efd8 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:53.099496 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 527/1038] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:52 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:53.101032 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3fcaca-6fe6-439c-bc6d-5cc95ceab7e5 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:53.117848 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:53.145720 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d973c92-c57a-4a50-8b1f-9431f79fea5c None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:53.146020 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d973c92-c57a-4a50-8b1f-9431f79fea5c None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:53.146276 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d973c92-c57a-4a50-8b1f-9431f79fea5c None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:53.147707 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 528/1039] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 12:02:53.152575 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d1cb6-6354-4993-a42d-4b67d39aaea5 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:53.152866 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d1cb6-6354-4993-a42d-4b67d39aaea5 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:53.153415 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d1cb6-6354-4993-a42d-4b67d39aaea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:53.172046 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-fd8d1cb6-6354-4993-a42d-4b67d39aaea5 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:53.173267 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-fd8d1cb6-6354-4993-a42d-4b67d39aaea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:53.209185 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3fcaca-6fe6-439c-bc6d-5cc95ceab7e5 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:53.244170 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 529/1040] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:53 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 12:02:53.246056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5ae521-0e1e-4b75-a7d2-ee8e2cedb9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:53.246669 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ae521-0e1e-4b75-a7d2-ee8e2cedb9ac None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:53.246956 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ae521-0e1e-4b75-a7d2-ee8e2cedb9ac None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:53.247239 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ae521-0e1e-4b75-a7d2-ee8e2cedb9ac None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:53.247960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5ae521-0e1e-4b75-a7d2-ee8e2cedb9ac None admin] RBAC: Adding query filter params (type=image) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:53.248272 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5ae521-0e1e-4b75-a7d2-ee8e2cedb9ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:53.249692 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5ae521-0e1e-4b75-a7d2-ee8e2cedb9ac None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:53.255785 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 512/1041] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:53 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:53.261390 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d1cb6-6354-4993-a42d-4b67d39aaea5 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:53.323906 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1751a8b7-5374-4063-91a7-e12dbfbb0878 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:53.324251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1751a8b7-5374-4063-91a7-e12dbfbb0878 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:53.324467 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1751a8b7-5374-4063-91a7-e12dbfbb0878 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:53.325965 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 513/1042] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:53.330761 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129629a7-89e2-4c1d-a0b2-6d6d65fac111 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:53.331058 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129629a7-89e2-4c1d-a0b2-6d6d65fac111 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:53.331339 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129629a7-89e2-4c1d-a0b2-6d6d65fac111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:53.332350 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:53.349871 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-129629a7-89e2-4c1d-a0b2-6d6d65fac111 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:53.351083 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-129629a7-89e2-4c1d-a0b2-6d6d65fac111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:53.364983 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d1cb6-6354-4993-a42d-4b67d39aaea5 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:53.401907 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a221cdb1-a657-4e0c-a58b-b5eeca5484aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:53.402517 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221cdb1-a657-4e0c-a58b-b5eeca5484aa None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:53.402792 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221cdb1-a657-4e0c-a58b-b5eeca5484aa None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:53.403395 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221cdb1-a657-4e0c-a58b-b5eeca5484aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:53.404129 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a221cdb1-a657-4e0c-a58b-b5eeca5484aa None admin] RBAC: Adding query filter params (interface=public, service_id=17c40593ee324114b9022516392cc11a, region_id=RegionOne) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:53.404492 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a221cdb1-a657-4e0c-a58b-b5eeca5484aa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:53.405862 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a221cdb1-a657-4e0c-a58b-b5eeca5484aa None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:53.416901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 530/1043] 10.222.0.98 () {56 vars in 1251 bytes} [Thu Aug 18 12:02:53 2022] GET /identity/v3/endpoints?service_id=17c40593ee324114b9022516392cc11a&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:53.417766 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a221cdb1-a657-4e0c-a58b-b5eeca5484aa None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:53.422260 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 514/1044] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:53 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:02:53.513943 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129629a7-89e2-4c1d-a0b2-6d6d65fac111 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:53.521975 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a221cdb1-a657-4e0c-a58b-b5eeca5484aa None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:53.522507 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:53.558579 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59532965-dad9-4ff2-af69-3542074f5fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:53.559203 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59532965-dad9-4ff2-af69-3542074f5fdb None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:53.559521 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59532965-dad9-4ff2-af69-3542074f5fdb None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:53.559804 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59532965-dad9-4ff2-af69-3542074f5fdb None admin] PATH_INFO: `/v3/services` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:53.560419 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59532965-dad9-4ff2-af69-3542074f5fdb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:53.561790 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59532965-dad9-4ff2-af69-3542074f5fdb None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:53.563667 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:53.569870 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 531/1045] 10.222.0.98 () {56 vars in 1087 bytes} [Thu Aug 18 12:02:53 2022] GET /identity/v3/services => generated 1957 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:02:53.625302 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129629a7-89e2-4c1d-a0b2-6d6d65fac111 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:53.664250 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431aa48-c1d0-4edf-99b2-9c428f89e62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:53.665259 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431aa48-c1d0-4edf-99b2-9c428f89e62b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:53.665728 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431aa48-c1d0-4edf-99b2-9c428f89e62b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:53.666185 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431aa48-c1d0-4edf-99b2-9c428f89e62b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:53.681257 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8431aa48-c1d0-4edf-99b2-9c428f89e62b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:02:53.686161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-8431aa48-c1d0-4edf-99b2-9c428f89e62b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 12:02:53.686161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:53.686161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:53.686161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:53.686161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:53.686161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:53.686161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:53.686161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:53.686161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:53.686161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:53.686161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:53.686161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:53.686161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:02:53.686161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:02:53.686161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:02:53.686161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:02:53.687361 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:02:53.687361 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:02:53.687361 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:02:53.687361 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:02:53.687361 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:53.687361 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:53.687361 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:53.687361 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:53.687361 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:53.687361 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:53.687361 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:53.687361 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:53.687361 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:53.687361 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:53.687361 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:53.687361 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:53.689005 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:53.689005 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:53.689005 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:53.689005 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:53.689005 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:02:53.689005 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:02:53.689005 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:02:53.689005 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:02:53.689005 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:02:53.689005 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:02:53.689005 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 12:02:53.689005 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:53.689005 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 515/1046] 10.222.0.98 () {56 vars in 1120 bytes} [Thu Aug 18 12:02:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:02:53.693960 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59532965-dad9-4ff2-af69-3542074f5fdb None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:53.694630 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431aa48-c1d0-4edf-99b2-9c428f89e62b None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:53.772889 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:53.814338 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59532965-dad9-4ff2-af69-3542074f5fdb None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:53.817323 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431aa48-c1d0-4edf-99b2-9c428f89e62b None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:53.851508 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b03ff0-fb26-49c0-8696-8479b4c2acc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:53.852125 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b03ff0-fb26-49c0-8696-8479b4c2acc0 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:53.852416 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b03ff0-fb26-49c0-8696-8479b4c2acc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:53.852695 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b03ff0-fb26-49c0-8696-8479b4c2acc0 None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:53.853363 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b03ff0-fb26-49c0-8696-8479b4c2acc0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:53.853672 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b03ff0-fb26-49c0-8696-8479b4c2acc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:53.854237 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e743f49d-4949-4490-8dc3-304f53ec6a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:53.854874 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743f49d-4949-4490-8dc3-304f53ec6a09 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:53.855054 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b03ff0-fb26-49c0-8696-8479b4c2acc0 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:53.855152 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743f49d-4949-4490-8dc3-304f53ec6a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:53.855442 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743f49d-4949-4490-8dc3-304f53ec6a09 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:53.856573 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e743f49d-4949-4490-8dc3-304f53ec6a09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:53.858005 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e743f49d-4949-4490-8dc3-304f53ec6a09 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:53.859877 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 516/1047] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:02:53.868655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 532/1048] 10.222.0.98 () {56 vars in 1115 bytes} [Thu Aug 18 12:02:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:53.868826 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e743f49d-4949-4490-8dc3-304f53ec6a09 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:53.869513 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b03ff0-fb26-49c0-8696-8479b4c2acc0 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:53.933636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:53.977039 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e743f49d-4949-4490-8dc3-304f53ec6a09 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:53.982159 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b03ff0-fb26-49c0-8696-8479b4c2acc0 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:54.012583 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad99b6-4ec1-4e0c-8993-5aa36a77d39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:54.013195 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad99b6-4ec1-4e0c-8993-5aa36a77d39f None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:54.013474 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad99b6-4ec1-4e0c-8993-5aa36a77d39f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:54.013748 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad99b6-4ec1-4e0c-8993-5aa36a77d39f None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:54.014403 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ad99b6-4ec1-4e0c-8993-5aa36a77d39f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:54.015859 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ad99b6-4ec1-4e0c-8993-5aa36a77d39f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:54.019004 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804db6c-eea0-4534-b945-d7c4161c4ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:54.020201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804db6c-eea0-4534-b945-d7c4161c4ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:54.020486 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804db6c-eea0-4534-b945-d7c4161c4ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:54.020764 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804db6c-eea0-4534-b945-d7c4161c4ce6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:54.024494 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:54.032887 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804db6c-eea0-4534-b945-d7c4161c4ce6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:54.034356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804db6c-eea0-4534-b945-d7c4161c4ce6 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:54.049168 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-4804db6c-eea0-4534-b945-d7c4161c4ce6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:54.049168 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:54.049168 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:54.049168 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:54.049168 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:54.049168 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:54.049168 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:54.049168 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:54.049168 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:54.049168 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:54.049168 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:54.049168 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:54.049168 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:54.049168 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:54.049168 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:54.049168 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:54.050383 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:54.050383 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:54.050383 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:54.050383 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:54.050383 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:54.050383 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:54.050383 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:54.050383 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:54.050383 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:54.050383 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:54.050383 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:54.050383 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:54.050383 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:54.050383 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:54.050383 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:54.050383 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:54.050383 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:54.051638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:54.051638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:54.051638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:54.051638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 533/1049] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:02:54.056068 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804db6c-eea0-4534-b945-d7c4161c4ce6 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:54.147636 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 517/1050] 10.222.0.98 () {60 vars in 1132 bytes} [Thu Aug 18 12:02:53 2022] POST /identity/v3/users => generated 285 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:02:54.179408 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804db6c-eea0-4534-b945-d7c4161c4ce6 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:54.195721 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:54.219693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3d3970-89ef-43b3-9af1-1e300673d8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:54.220404 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d3970-89ef-43b3-9af1-1e300673d8e0 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:54.220754 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d3970-89ef-43b3-9af1-1e300673d8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:54.221104 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d3970-89ef-43b3-9af1-1e300673d8e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:54.222275 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3d3970-89ef-43b3-9af1-1e300673d8e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:54.223537 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3d3970-89ef-43b3-9af1-1e300673d8e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:54.225481 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3d3970-89ef-43b3-9af1-1e300673d8e0 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:54.240172 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 534/1051] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:02:54.245768 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad99b6-4ec1-4e0c-8993-5aa36a77d39f None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:54.354285 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad99b6-4ec1-4e0c-8993-5aa36a77d39f None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:54.389363 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b2b4b-c669-4042-a0ee-7e8a83a06912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:54.390003 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b2b4b-c669-4042-a0ee-7e8a83a06912 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:54.390383 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b2b4b-c669-4042-a0ee-7e8a83a06912 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:54.390554 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b2b4b-c669-4042-a0ee-7e8a83a06912 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:54.399623 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79b2b4b-c669-4042-a0ee-7e8a83a06912 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:54.401069 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79b2b4b-c669-4042-a0ee-7e8a83a06912 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:54.412470 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-c79b2b4b-c669-4042-a0ee-7e8a83a06912 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 12:02:54.412470 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:54.412470 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:54.412470 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:54.412470 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:54.412470 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:54.412470 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:54.412470 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:54.412470 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:54.412470 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:54.412470 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:54.412470 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:54.412470 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 12:02:54.412470 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 12:02:54.412470 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 12:02:54.412470 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 12:02:54.413410 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:54.413410 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:54.413410 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:02:54.413410 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:54.413410 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:02:54.413410 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:54.413410 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:54.413410 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:54.413410 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:54.413410 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:54.413410 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:54.413410 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:54.413410 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:54.413410 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:54.413410 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:54.413410 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:54.413410 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:54.413410 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:54.414362 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:54.414362 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:54.414362 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 12:02:54.414362 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 12:02:54.414362 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 12:02:54.414362 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 12:02:54.414362 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 12:02:54.414362 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 12:02:54.414362 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 12:02:54.414362 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:54.414362 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 518/1052] 10.222.0.98 () {56 vars in 1093 bytes} [Thu Aug 18 12:02:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:54.419527 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3d3970-89ef-43b3-9af1-1e300673d8e0 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:54.530796 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3d3970-89ef-43b3-9af1-1e300673d8e0 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:54.565335 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de9596-ecf8-4da3-8f53-7b66594fc78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:54.565957 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de9596-ecf8-4da3-8f53-7b66594fc78a None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:54.566219 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de9596-ecf8-4da3-8f53-7b66594fc78a None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:54.566493 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de9596-ecf8-4da3-8f53-7b66594fc78a None admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:54.568792 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41de9596-ecf8-4da3-8f53-7b66594fc78a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:54.569103 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41de9596-ecf8-4da3-8f53-7b66594fc78a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:54.570601 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41de9596-ecf8-4da3-8f53-7b66594fc78a None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:54.590020 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 535/1053] 10.222.0.98 () {56 vars in 1133 bytes} [Thu Aug 18 12:02:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:54.595402 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b2b4b-c669-4042-a0ee-7e8a83a06912 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:54.647057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:54.706720 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b2b4b-c669-4042-a0ee-7e8a83a06912 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:54.741409 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a514867-79aa-44e0-a01f-6fbac5911ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:54.742337 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a514867-79aa-44e0-a01f-6fbac5911ccb None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:54.742804 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a514867-79aa-44e0-a01f-6fbac5911ccb None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:54.743370 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a514867-79aa-44e0-a01f-6fbac5911ccb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:54.755065 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a514867-79aa-44e0-a01f-6fbac5911ccb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:54.756774 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a514867-79aa-44e0-a01f-6fbac5911ccb None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:54.772894 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-3a514867-79aa-44e0-a01f-6fbac5911ccb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:54.772894 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:54.772894 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:54.772894 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:54.772894 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:54.772894 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:54.772894 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:54.772894 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:54.772894 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:54.772894 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:54.772894 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:54.772894 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:54.772894 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:54.772894 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:54.772894 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:54.772894 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:54.773886 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:54.773886 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:54.773886 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:54.773886 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:54.773886 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:54.773886 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:54.773886 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:54.773886 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:54.773886 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:54.773886 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:54.773886 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:54.773886 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:54.773886 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:54.773886 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:54.773886 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:54.773886 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:54.773886 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:54.774785 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:54.774785 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:54.774785 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:54.781364 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 519/1054] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:54.782356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de9596-ecf8-4da3-8f53-7b66594fc78a None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:54.815370 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:54.887978 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de9596-ecf8-4da3-8f53-7b66594fc78a None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:54.922712 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71acf82-cb34-4db6-aeb7-14d1a27be93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:54.923631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71acf82-cb34-4db6-aeb7-14d1a27be93b None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:54.924100 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71acf82-cb34-4db6-aeb7-14d1a27be93b None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:54.924557 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71acf82-cb34-4db6-aeb7-14d1a27be93b None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:54.925379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71acf82-cb34-4db6-aeb7-14d1a27be93b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:54.925884 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71acf82-cb34-4db6-aeb7-14d1a27be93b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:54.928088 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71acf82-cb34-4db6-aeb7-14d1a27be93b None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:54.944946 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 536/1055] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:02:54.946204 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a514867-79aa-44e0-a01f-6fbac5911ccb None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:55.048967 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a514867-79aa-44e0-a01f-6fbac5911ccb None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:55.051752 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:55.085292 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3423e9bd-7e30-4c92-8046-b5ee7f8b3e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:55.085933 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423e9bd-7e30-4c92-8046-b5ee7f8b3e9f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:55.086215 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423e9bd-7e30-4c92-8046-b5ee7f8b3e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:55.086490 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423e9bd-7e30-4c92-8046-b5ee7f8b3e9f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:55.093394 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3423e9bd-7e30-4c92-8046-b5ee7f8b3e9f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:55.094842 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3423e9bd-7e30-4c92-8046-b5ee7f8b3e9f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:55.107577 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-3423e9bd-7e30-4c92-8046-b5ee7f8b3e9f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:55.107577 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:55.107577 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:55.107577 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:55.107577 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:55.107577 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:55.107577 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:55.107577 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:55.107577 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:55.107577 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:55.107577 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:55.107577 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:55.107577 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 12:02:55.107577 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 12:02:55.107577 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 12:02:55.108506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 12:02:55.108506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:55.108506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:55.108506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:55.108506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:55.108506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:55.108506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:55.108506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:55.108506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:55.108506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:55.108506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:55.108506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:55.108506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:55.108506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:55.108506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:55.108506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:55.108506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:55.109437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 12:02:55.109437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 12:02:55.109437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:02:55.109437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:02:55.109437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:02:55.109437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:02:55.109437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:55.109437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:55.111769 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 520/1056] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:02:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:02:55.115052 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71acf82-cb34-4db6-aeb7-14d1a27be93b None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:55.214556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:55.224806 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71acf82-cb34-4db6-aeb7-14d1a27be93b None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:55.258974 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1e24e-0f0b-4a90-8a3a-412af39a89af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:55.259624 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1e24e-0f0b-4a90-8a3a-412af39a89af None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:55.259917 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1e24e-0f0b-4a90-8a3a-412af39a89af None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:55.260193 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1e24e-0f0b-4a90-8a3a-412af39a89af None admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:55.260883 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad1e24e-0f0b-4a90-8a3a-412af39a89af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:55.261195 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad1e24e-0f0b-4a90-8a3a-412af39a89af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:55.262626 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad1e24e-0f0b-4a90-8a3a-412af39a89af None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:55.275059 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 537/1057] 10.222.0.98 () {56 vars in 1148 bytes} [Thu Aug 18 12:02:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:02:55.281108 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3423e9bd-7e30-4c92-8046-b5ee7f8b3e9f None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:55.385296 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3423e9bd-7e30-4c92-8046-b5ee7f8b3e9f None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:55.419302 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505bdfe8-d0bf-497c-ac74-92195f967004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:55.419993 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bdfe8-d0bf-497c-ac74-92195f967004 None admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:55.420281 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bdfe8-d0bf-497c-ac74-92195f967004 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:55.420561 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bdfe8-d0bf-497c-ac74-92195f967004 None admin] PATH_INFO: `/v3/projects/6f676a5169464d83af4267db25025728/users/171c72a6b29e418bbb4ea5c6faffe3b3/roles/9bcab8f185d94480b7fb843825dfb542` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:55.422693 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505bdfe8-d0bf-497c-ac74-92195f967004 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f676a5169464d83af4267db25025728, user_id=171c72a6b29e418bbb4ea5c6faffe3b3, role_id=9bcab8f185d94480b7fb843825dfb542)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:55.424157 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505bdfe8-d0bf-497c-ac74-92195f967004 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:55.509495 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 521/1058] 10.222.0.98 () {58 vars in 1439 bytes} [Thu Aug 18 12:02:55 2022] PUT /identity/v3/projects/6f676a5169464d83af4267db25025728/users/171c72a6b29e418bbb4ea5c6faffe3b3/roles/9bcab8f185d94480b7fb843825dfb542 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:02:55.526206 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae51d25c-5aa4-4135-9688-f58cddb9c45a None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:55.526746 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae51d25c-5aa4-4135-9688-f58cddb9c45a None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:55.526746 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae51d25c-5aa4-4135-9688-f58cddb9c45a None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:55.528743 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 538/1059] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:55.534584 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcddb9bf-43b3-4e34-879c-55a038ff9b9b None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:55.534941 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcddb9bf-43b3-4e34-879c-55a038ff9b9b None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:55.535160 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcddb9bf-43b3-4e34-879c-55a038ff9b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:55.555697 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-fcddb9bf-43b3-4e34-879c-55a038ff9b9b None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:55.556946 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-fcddb9bf-43b3-4e34-879c-55a038ff9b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:55.577224 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:55.639395 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 522/1060] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:55 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:02:55.906105 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae51d25c-5aa4-4135-9688-f58cddb9c45a None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:56.016272 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae51d25c-5aa4-4135-9688-f58cddb9c45a None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:56.052381 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f741cabf-3b58-4b56-85be-8a555391ccb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:56.053019 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f741cabf-3b58-4b56-85be-8a555391ccb3 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:56.053303 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f741cabf-3b58-4b56-85be-8a555391ccb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:56.053583 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f741cabf-3b58-4b56-85be-8a555391ccb3 None admin] PATH_INFO: `/v3/services/image` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:56.054173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f741cabf-3b58-4b56-85be-8a555391ccb3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:56.055580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f741cabf-3b58-4b56-85be-8a555391ccb3 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:56.064376 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-f741cabf-3b58-4b56-85be-8a555391ccb3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 12:02:56.064376 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:56.064376 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 12:02:56.064376 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 12:02:56.064376 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 12:02:56.064376 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 12:02:56.064376 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 12:02:56.064376 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:56.064376 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 12:02:56.064376 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:56.064376 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 12:02:56.064376 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:56.064376 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:56.064376 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:56.064376 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:56.064376 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:56.064376 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:56.065413 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:56.065413 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:56.065413 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:56.065413 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:56.065413 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:56.065413 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:56.065413 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 12:02:56.065413 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 12:02:56.065413 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 12:02:56.065413 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 12:02:56.065413 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:56.065413 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:56.065413 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:56.065413 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:56.065413 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:56.065413 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:56.066262 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:56.066262 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:56.066262 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:56.066262 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:56.066262 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:56.066262 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:56.066262 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:56.066262 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:56.066262 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:56.066262 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:56.066262 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 12:02:56.066262 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:02:56.066262 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 18 12:02:56.066262 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:56.066262 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 539/1061] 10.222.0.98 () {56 vars in 1105 bytes} [Thu Aug 18 12:02:55 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:56.073829 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcddb9bf-43b3-4e34-879c-55a038ff9b9b None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:56.173734 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d674cb8-20a0-4fd4-95b2-0c381b36f295 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:56.174037 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d674cb8-20a0-4fd4-95b2-0c381b36f295 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:56.174236 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d674cb8-20a0-4fd4-95b2-0c381b36f295 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:56.179558 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 540/1062] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:56.179750 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcddb9bf-43b3-4e34-879c-55a038ff9b9b None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:56.183375 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded7dbd-f765-4cb2-8754-3fc5d4041eef None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:56.183796 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded7dbd-f765-4cb2-8754-3fc5d4041eef None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:56.185081 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded7dbd-f765-4cb2-8754-3fc5d4041eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:56.215310 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-2ded7dbd-f765-4cb2-8754-3fc5d4041eef None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:56.216626 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-2ded7dbd-f765-4cb2-8754-3fc5d4041eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:56.218933 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c217569-46f0-40d0-82f3-e57c1cc76469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:56.219584 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c217569-46f0-40d0-82f3-e57c1cc76469 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:56.219871 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c217569-46f0-40d0-82f3-e57c1cc76469 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:56.220148 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c217569-46f0-40d0-82f3-e57c1cc76469 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:56.220801 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c217569-46f0-40d0-82f3-e57c1cc76469 None admin] RBAC: Adding query filter params (name=image) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:56.221111 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c217569-46f0-40d0-82f3-e57c1cc76469 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:56.222546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c217569-46f0-40d0-82f3-e57c1cc76469 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:56.228097 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 523/1063] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:56.233168 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:56.233659 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c217569-46f0-40d0-82f3-e57c1cc76469 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:56.255735 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:56.287324 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 541/1064] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:56 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 12:02:56.346245 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c217569-46f0-40d0-82f3-e57c1cc76469 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:56.384882 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cae6f2-a482-474e-91a0-6e1237c8b909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:56.385491 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cae6f2-a482-474e-91a0-6e1237c8b909 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:56.385773 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cae6f2-a482-474e-91a0-6e1237c8b909 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:56.386048 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cae6f2-a482-474e-91a0-6e1237c8b909 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:56.386678 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cae6f2-a482-474e-91a0-6e1237c8b909 None admin] RBAC: Adding query filter params (type=image) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:56.386982 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cae6f2-a482-474e-91a0-6e1237c8b909 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:56.388390 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cae6f2-a482-474e-91a0-6e1237c8b909 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:56.393569 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 524/1065] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:02:56.399278 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ded7dbd-f765-4cb2-8754-3fc5d4041eef None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:56.503023 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ded7dbd-f765-4cb2-8754-3fc5d4041eef None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:56.544411 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914d615-b762-48cd-b18b-41864021c1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:56.545154 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914d615-b762-48cd-b18b-41864021c1a5 None admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:56.545382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914d615-b762-48cd-b18b-41864021c1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:56.545563 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914d615-b762-48cd-b18b-41864021c1a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:56.546240 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3914d615-b762-48cd-b18b-41864021c1a5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:56.547643 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3914d615-b762-48cd-b18b-41864021c1a5 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:56.564272 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cae6f2-a482-474e-91a0-6e1237c8b909 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:56.631838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 542/1066] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:02:56 2022] POST /identity/v3/endpoints => generated 327 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:02:56.656345 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:56.685923 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cae6f2-a482-474e-91a0-6e1237c8b909 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:56.730054 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ddffd4-4333-4285-b39a-ad65d2d7e39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:56.730694 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ddffd4-4333-4285-b39a-ad65d2d7e39c None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:56.730977 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ddffd4-4333-4285-b39a-ad65d2d7e39c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:56.731251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ddffd4-4333-4285-b39a-ad65d2d7e39c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:56.743452 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ddffd4-4333-4285-b39a-ad65d2d7e39c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:02:56.748259 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-b8ddffd4-4333-4285-b39a-ad65d2d7e39c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:02:56.748259 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:56.748259 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:56.748259 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:56.748259 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:56.748259 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:56.748259 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:56.748259 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:56.748259 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:56.748259 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:56.748259 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:56.748259 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:56.748259 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:02:56.748259 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:02:56.748259 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:02:56.748259 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:02:56.749225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:02:56.749225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:02:56.749225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:02:56.749225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:02:56.749225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:56.749225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:56.749225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:56.749225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:56.749225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:56.749225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:56.749225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:56.749225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:56.749225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:56.749225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:56.749225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:56.749225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:56.752437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:56.752437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:56.752437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:56.752437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:56.752437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:02:56.752437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:02:56.752437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:02:56.752437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:02:56.752437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:02:56.752437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:02:56.752437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:02:56.752437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:56.752437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 525/1067] 10.222.0.98 () {56 vars in 1096 bytes} [Thu Aug 18 12:02:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:56.756093 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914d615-b762-48cd-b18b-41864021c1a5 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:56.759230 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:56.868154 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914d615-b762-48cd-b18b-41864021c1a5 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:56.904871 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae90d69-aefb-4729-874c-a39295ff697c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:56.905769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae90d69-aefb-4729-874c-a39295ff697c None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:56.906215 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae90d69-aefb-4729-874c-a39295ff697c None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:56.906643 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae90d69-aefb-4729-874c-a39295ff697c None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:56.907497 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae90d69-aefb-4729-874c-a39295ff697c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:56.907970 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae90d69-aefb-4729-874c-a39295ff697c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:56.909550 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae90d69-aefb-4729-874c-a39295ff697c None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:56.919612 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 543/1068] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:02:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:02:56.925244 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ddffd4-4333-4285-b39a-ad65d2d7e39c None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:57.032194 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ddffd4-4333-4285-b39a-ad65d2d7e39c None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:57.067435 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb1a8f8-f55c-44cd-91ea-922606a0aa85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:57.068087 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1a8f8-f55c-44cd-91ea-922606a0aa85 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:57.068451 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1a8f8-f55c-44cd-91ea-922606a0aa85 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:57.068643 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1a8f8-f55c-44cd-91ea-922606a0aa85 None admin] PATH_INFO: `/v3/users/830f8a6395f9481c84654492406bb3bc` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:57.071384 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:57.084046 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb1a8f8-f55c-44cd-91ea-922606a0aa85 None admin] RBAC: Authorizing `identity:get_user(user_id=830f8a6395f9481c84654492406bb3bc)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:57.085499 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb1a8f8-f55c-44cd-91ea-922606a0aa85 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:57.092308 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 526/1069] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:02:56 2022] GET /identity/v3/users/830f8a6395f9481c84654492406bb3bc => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:02:57.093302 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae90d69-aefb-4729-874c-a39295ff697c None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:57.164164 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:57.203130 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae90d69-aefb-4729-874c-a39295ff697c None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:57.238415 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ea023-da38-4f63-a6f2-5f55ddba3767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:57.239778 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ea023-da38-4f63-a6f2-5f55ddba3767 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:57.239957 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ea023-da38-4f63-a6f2-5f55ddba3767 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:57.240219 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ea023-da38-4f63-a6f2-5f55ddba3767 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:57.309014 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6ea023-da38-4f63-a6f2-5f55ddba3767 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:57.310379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6ea023-da38-4f63-a6f2-5f55ddba3767 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:57.372525 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:57.412755 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 544/1070] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:02:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 18 12:02:57.418042 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb1a8f8-f55c-44cd-91ea-922606a0aa85 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:57.432238 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89ce14-f2a1-47dc-bfb8-9368d5a42f43 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:57.432533 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89ce14-f2a1-47dc-bfb8-9368d5a42f43 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:57.432816 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89ce14-f2a1-47dc-bfb8-9368d5a42f43 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:57.434297 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 545/1071] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 12:02:57.439960 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad32bd4-a9d4-466d-8159-5377fbacf674 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:57.440248 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad32bd4-a9d4-466d-8159-5377fbacf674 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:57.440569 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad32bd4-a9d4-466d-8159-5377fbacf674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:57.458485 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-cad32bd4-a9d4-466d-8159-5377fbacf674 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:57.459702 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-cad32bd4-a9d4-466d-8159-5377fbacf674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:57.537116 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 546/1072] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 12:02:57.540810 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb1a8f8-f55c-44cd-91ea-922606a0aa85 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:57.580801 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343cedc-caa2-4d5e-a4ae-f275bdbb9d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:57.581262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343cedc-caa2-4d5e-a4ae-f275bdbb9d43 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:57.581547 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343cedc-caa2-4d5e-a4ae-f275bdbb9d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:57.581827 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343cedc-caa2-4d5e-a4ae-f275bdbb9d43 None admin] PATH_INFO: `/v3/projects/fafb788ca7ac448e903522b57eb1eb5e` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:57.582986 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343cedc-caa2-4d5e-a4ae-f275bdbb9d43 None admin] RBAC: Authorizing `identity:get_project(project_id=fafb788ca7ac448e903522b57eb1eb5e)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:57.584482 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343cedc-caa2-4d5e-a4ae-f275bdbb9d43 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:57.591113 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 527/1073] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:02:57 2022] GET /identity/v3/projects/fafb788ca7ac448e903522b57eb1eb5e => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:57.592304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad32bd4-a9d4-466d-8159-5377fbacf674 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:57.698622 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:57.705298 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad32bd4-a9d4-466d-8159-5377fbacf674 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:57.743247 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc5985-19c1-4ba9-b481-77d0322f07f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:57.743891 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc5985-19c1-4ba9-b481-77d0322f07f4 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:57.744177 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc5985-19c1-4ba9-b481-77d0322f07f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:57.744473 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc5985-19c1-4ba9-b481-77d0322f07f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:57.745192 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fc5985-19c1-4ba9-b481-77d0322f07f4 None admin] RBAC: Adding query filter params (role.id=c53a0e13b58642f291c856efe1f0b87d, scope.project.id=fafb788ca7ac448e903522b57eb1eb5e, user.id=830f8a6395f9481c84654492406bb3bc) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:57.745505 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fc5985-19c1-4ba9-b481-77d0322f07f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:57.746951 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fc5985-19c1-4ba9-b481-77d0322f07f4 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:57.759735 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 547/1074] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:02:57 2022] GET /identity/v3/role_assignments?user.id=830f8a6395f9481c84654492406bb3bc&scope.project.id=fafb788ca7ac448e903522b57eb1eb5e&role.id=c53a0e13b58642f291c856efe1f0b87d => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:02:57.809011 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343cedc-caa2-4d5e-a4ae-f275bdbb9d43 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:57.883606 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:57.927493 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343cedc-caa2-4d5e-a4ae-f275bdbb9d43 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:57.961740 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55eb716-d9ab-4e73-83bf-10da9285594f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:57.962414 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55eb716-d9ab-4e73-83bf-10da9285594f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:57.962699 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55eb716-d9ab-4e73-83bf-10da9285594f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:57.962967 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55eb716-d9ab-4e73-83bf-10da9285594f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:57.974070 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55eb716-d9ab-4e73-83bf-10da9285594f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:02:57.978858 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-f55eb716-d9ab-4e73-83bf-10da9285594f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 12:02:57.978858 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:57.978858 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:57.978858 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:57.978858 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:57.978858 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:57.978858 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:57.978858 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:57.978858 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:57.978858 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:57.978858 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:57.978858 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:57.978858 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:02:57.978858 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:02:57.978858 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:02:57.978858 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:02:57.980711 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:02:57.980711 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:02:57.980711 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:02:57.980711 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:02:57.980711 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:57.980711 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:57.980711 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:57.980711 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:57.980711 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:57.980711 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:57.980711 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:57.980711 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:57.980711 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:57.980711 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:57.980711 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:57.980711 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:57.982707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:57.982707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:57.982707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:57.982707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:57.982707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:02:57.982707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:02:57.982707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:02:57.982707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:02:57.982707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:02:57.982707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:02:57.982707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 18 12:02:57.982707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:57.982707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 528/1075] 10.222.0.98 () {56 vars in 1120 bytes} [Thu Aug 18 12:02:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:57.988376 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc5985-19c1-4ba9-b481-77d0322f07f4 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:58.097377 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc5985-19c1-4ba9-b481-77d0322f07f4 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:58.133212 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb615b-e9ab-445c-b785-6a46bf796d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:58.133889 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb615b-e9ab-445c-b785-6a46bf796d83 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:58.134116 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb615b-e9ab-445c-b785-6a46bf796d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:58.134399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb615b-e9ab-445c-b785-6a46bf796d83 None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:58.135124 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccb615b-e9ab-445c-b785-6a46bf796d83 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:58.135463 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccb615b-e9ab-445c-b785-6a46bf796d83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:58.136846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccb615b-e9ab-445c-b785-6a46bf796d83 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:58.150505 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 548/1076] 10.222.0.98 () {56 vars in 1115 bytes} [Thu Aug 18 12:02:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:58.152312 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55eb716-d9ab-4e73-83bf-10da9285594f None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:58.268850 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55eb716-d9ab-4e73-83bf-10da9285594f None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:58.292869 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:58.304834 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee6ad1-152c-4f68-99f9-42cdbe958074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:58.305478 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee6ad1-152c-4f68-99f9-42cdbe958074 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:58.305763 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee6ad1-152c-4f68-99f9-42cdbe958074 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:58.307253 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee6ad1-152c-4f68-99f9-42cdbe958074 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:58.316250 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee6ad1-152c-4f68-99f9-42cdbe958074 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:58.318301 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee6ad1-152c-4f68-99f9-42cdbe958074 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:58.333846 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-65ee6ad1-152c-4f68-99f9-42cdbe958074 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:58.333846 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:58.333846 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:58.333846 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:58.333846 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:58.333846 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:58.333846 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:58.333846 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:58.333846 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:58.333846 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:58.333846 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:58.333846 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:58.333846 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:58.333846 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:58.333846 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:58.333846 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:58.335036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:58.335036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:58.335036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:58.335036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:58.335036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:58.335036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:58.335036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:58.335036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:58.335036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:58.335036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:58.335036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:58.335036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:58.335036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:58.335036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:58.335036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:58.335036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:58.335036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:58.336251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:58.336251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:58.336251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:58.342973 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 529/1077] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:58.344376 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb615b-e9ab-445c-b785-6a46bf796d83 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:58.368597 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:58.449290 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb615b-e9ab-445c-b785-6a46bf796d83 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:58.482040 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffed912-f8a4-49ca-aa52-e97a9dcf65a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:58.482640 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffed912-f8a4-49ca-aa52-e97a9dcf65a4 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:58.482918 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffed912-f8a4-49ca-aa52-e97a9dcf65a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:58.483191 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffed912-f8a4-49ca-aa52-e97a9dcf65a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:58.483854 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffed912-f8a4-49ca-aa52-e97a9dcf65a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:58.484158 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffed912-f8a4-49ca-aa52-e97a9dcf65a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:58.485508 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffed912-f8a4-49ca-aa52-e97a9dcf65a4 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:58.497973 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 549/1078] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:58.503540 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee6ad1-152c-4f68-99f9-42cdbe958074 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:58.574462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213b97e-d835-4676-b270-19c0fbee7cdc None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:58.574845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213b97e-d835-4676-b270-19c0fbee7cdc None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:58.575033 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213b97e-d835-4676-b270-19c0fbee7cdc None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:58.576541 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 550/1079] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 12:02:58.581194 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987e175-ddff-499c-8cf8-2ded34126601 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:58.581479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987e175-ddff-499c-8cf8-2ded34126601 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:58.581757 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987e175-ddff-499c-8cf8-2ded34126601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:58.600764 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-9987e175-ddff-499c-8cf8-2ded34126601 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:58.601973 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-9987e175-ddff-499c-8cf8-2ded34126601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:58.618941 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee6ad1-152c-4f68-99f9-42cdbe958074 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:58.661263 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2ea3a-450e-49e6-b48a-cc27d03c6935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:58.661937 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2ea3a-450e-49e6-b48a-cc27d03c6935 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:58.662220 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2ea3a-450e-49e6-b48a-cc27d03c6935 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:58.662490 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2ea3a-450e-49e6-b48a-cc27d03c6935 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:58.670833 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd2ea3a-450e-49e6-b48a-cc27d03c6935 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:58.672328 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd2ea3a-450e-49e6-b48a-cc27d03c6935 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:58.682748 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 551/1080] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 12:02:58.685826 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-0cd2ea3a-450e-49e6-b48a-cc27d03c6935 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 18 12:02:58.685826 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:58.685826 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:58.685826 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:58.685826 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:58.685826 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:58.685826 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:58.685826 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:58.685826 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:58.685826 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:58.685826 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:58.685826 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:58.685826 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 12:02:58.685826 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 12:02:58.685826 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 12:02:58.685826 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 12:02:58.686901 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:58.686901 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:58.686901 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:02:58.686901 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:58.686901 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:02:58.686901 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:02:58.686901 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:58.686901 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:58.686901 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:58.686901 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:58.686901 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:58.686901 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:58.686901 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:58.686901 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:58.686901 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:58.686901 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:58.686901 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:58.686901 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:58.688550 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:58.688550 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:58.688550 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 12:02:58.688550 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 12:02:58.688550 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 12:02:58.688550 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 12:02:58.688550 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 12:02:58.688550 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 12:02:58.688550 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 18 12:02:58.688550 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:58.688550 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 530/1081] 10.222.0.98 () {56 vars in 1093 bytes} [Thu Aug 18 12:02:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:58.693999 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987e175-ddff-499c-8cf8-2ded34126601 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:58.707147 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:58.806806 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987e175-ddff-499c-8cf8-2ded34126601 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:58.845002 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974abd6c-2c20-4202-91f5-aa3537ffa85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:58.845687 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974abd6c-2c20-4202-91f5-aa3537ffa85b None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:58.845972 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974abd6c-2c20-4202-91f5-aa3537ffa85b None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:58.846246 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974abd6c-2c20-4202-91f5-aa3537ffa85b None admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:58.847018 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974abd6c-2c20-4202-91f5-aa3537ffa85b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:58.847328 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974abd6c-2c20-4202-91f5-aa3537ffa85b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:58.849347 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974abd6c-2c20-4202-91f5-aa3537ffa85b None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:58.868753 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 552/1082] 10.222.0.98 () {56 vars in 1133 bytes} [Thu Aug 18 12:02:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:02:58.874340 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2ea3a-450e-49e6-b48a-cc27d03c6935 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:58.913398 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:58.956069 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974abd6c-2c20-4202-91f5-aa3537ffa85b None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:58.984148 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2ea3a-450e-49e6-b48a-cc27d03c6935 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:59.022374 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc79334-61d9-4ea9-b620-dec0cfeb49e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:59.022981 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc79334-61d9-4ea9-b620-dec0cfeb49e6 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:59.023340 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc79334-61d9-4ea9-b620-dec0cfeb49e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:59.023650 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc79334-61d9-4ea9-b620-dec0cfeb49e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:59.029704 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:59.068476 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974abd6c-2c20-4202-91f5-aa3537ffa85b None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:59.099762 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc79334-61d9-4ea9-b620-dec0cfeb49e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:59.101308 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc79334-61d9-4ea9-b620-dec0cfeb49e6 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:59.109437 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9bbc2b-5bfa-4bcb-b28a-2038fc3ec895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:59.110057 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9bbc2b-5bfa-4bcb-b28a-2038fc3ec895 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:59.110343 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9bbc2b-5bfa-4bcb-b28a-2038fc3ec895 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:59.110619 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9bbc2b-5bfa-4bcb-b28a-2038fc3ec895 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:59.136665 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:59.189709 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9bbc2b-5bfa-4bcb-b28a-2038fc3ec895 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:59.191525 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9bbc2b-5bfa-4bcb-b28a-2038fc3ec895 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:59.216862 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 531/1083] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:02:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 12:02:59.218108 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc79334-61d9-4ea9-b620-dec0cfeb49e6 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:59.269566 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:59.306924 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 553/1084] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:02:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 18 12:02:59.312779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9bbc2b-5bfa-4bcb-b28a-2038fc3ec895 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:59.336290 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc79334-61d9-4ea9-b620-dec0cfeb49e6 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:59.377011 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d8e6a-4beb-488c-bbfe-96ff31504e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:59.377640 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d8e6a-4beb-488c-bbfe-96ff31504e4f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:59.378045 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d8e6a-4beb-488c-bbfe-96ff31504e4f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:59.378325 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d8e6a-4beb-488c-bbfe-96ff31504e4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:59.380604 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:59.391563 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302d8e6a-4beb-488c-bbfe-96ff31504e4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:59.397227 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302d8e6a-4beb-488c-bbfe-96ff31504e4f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:59.410036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-302d8e6a-4beb-488c-bbfe-96ff31504e4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:59.410036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:59.410036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:59.410036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:59.410036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:59.410036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:59.410036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:59.410036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:59.410036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:59.410036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:59.410036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:59.410036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:59.410036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:59.410036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:59.410036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:59.410036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:59.411850 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:59.411850 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:59.411850 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:59.411850 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:59.411850 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:59.411850 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:59.411850 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:59.411850 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:59.411850 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:59.411850 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:59.411850 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:59.411850 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:59.411850 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:59.411850 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:59.411850 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:59.411850 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:59.411850 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:59.412751 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:59.412751 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:59.412751 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:59.412751 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 532/1085] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:02:59.417496 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d8e6a-4beb-488c-bbfe-96ff31504e4f None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:59.425452 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9bbc2b-5bfa-4bcb-b28a-2038fc3ec895 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:59.464214 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1064b7a4-4259-4139-a5cc-6499b11f1b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:59.464848 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064b7a4-4259-4139-a5cc-6499b11f1b48 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:59.465135 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064b7a4-4259-4139-a5cc-6499b11f1b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:59.465415 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064b7a4-4259-4139-a5cc-6499b11f1b48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:59.477898 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1064b7a4-4259-4139-a5cc-6499b11f1b48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:59.479482 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1064b7a4-4259-4139-a5cc-6499b11f1b48 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:59.496187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-1064b7a4-4259-4139-a5cc-6499b11f1b48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:59.496187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:59.496187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:59.496187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:59.496187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:59.496187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:59.496187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:59.496187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:59.496187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:59.496187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:59.496187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:59.496187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:59.496187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:02:59.496187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:02:59.496187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:02:59.496187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:02:59.497200 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:59.497200 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:59.497200 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:59.497200 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:59.497200 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:59.497200 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:59.497200 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:59.497200 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:59.497200 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:59.497200 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:59.497200 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:59.497200 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:59.497200 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:59.497200 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:59.497200 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:59.497200 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:59.497200 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:02:59.498118 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:02:59.498118 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:02:59.498118 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:02:59.499579 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 554/1086] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:02:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:02:59.503146 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1064b7a4-4259-4139-a5cc-6499b11f1b48 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:59.527995 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d8e6a-4beb-488c-bbfe-96ff31504e4f None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:59.534891 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:59.566801 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3dc06-b2bc-4b47-a299-0065cdf7074d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:59.568104 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3dc06-b2bc-4b47-a299-0065cdf7074d None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:59.568391 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3dc06-b2bc-4b47-a299-0065cdf7074d None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:59.568717 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3dc06-b2bc-4b47-a299-0065cdf7074d None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:59.569441 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d3dc06-b2bc-4b47-a299-0065cdf7074d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:59.569750 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d3dc06-b2bc-4b47-a299-0065cdf7074d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:59.571122 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d3dc06-b2bc-4b47-a299-0065cdf7074d None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:59.584430 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 533/1087] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:02:59.589960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3dc06-b2bc-4b47-a299-0065cdf7074d None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:59.622225 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1064b7a4-4259-4139-a5cc-6499b11f1b48 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:59.640799 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:59.661866 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55edc5d5-3931-4fac-be29-59d8319c271d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:59.662580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55edc5d5-3931-4fac-be29-59d8319c271d None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:59.662860 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55edc5d5-3931-4fac-be29-59d8319c271d None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:59.663133 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55edc5d5-3931-4fac-be29-59d8319c271d None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:59.663912 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55edc5d5-3931-4fac-be29-59d8319c271d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:59.664181 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55edc5d5-3931-4fac-be29-59d8319c271d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:59.665519 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55edc5d5-3931-4fac-be29-59d8319c271d None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:59.679746 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 555/1088] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:02:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:02:59.681144 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bd788-e400-4b6e-8b37-f244270b7f5e None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:59.681399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bd788-e400-4b6e-8b37-f244270b7f5e None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:59.681683 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bd788-e400-4b6e-8b37-f244270b7f5e None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:59.683128 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 556/1089] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:02:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:02:59.688778 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d26f69-2059-4f06-9f2d-8015be8a5ee8 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:59.689074 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d26f69-2059-4f06-9f2d-8015be8a5ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:59.689357 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d26f69-2059-4f06-9f2d-8015be8a5ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:59.711871 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-99d26f69-2059-4f06-9f2d-8015be8a5ee8 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:02:59.713135 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-99d26f69-2059-4f06-9f2d-8015be8a5ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:02:59.717327 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3dc06-b2bc-4b47-a299-0065cdf7074d None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:59.757989 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed72395-1131-4a36-a13b-23e79ec17dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:59.758643 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed72395-1131-4a36-a13b-23e79ec17dc0 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:59.758923 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed72395-1131-4a36-a13b-23e79ec17dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:59.759196 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed72395-1131-4a36-a13b-23e79ec17dc0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:59.767117 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed72395-1131-4a36-a13b-23e79ec17dc0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:59.768645 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed72395-1131-4a36-a13b-23e79ec17dc0 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:59.781311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-fed72395-1131-4a36-a13b-23e79ec17dc0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:59.781311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:02:59.781311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:02:59.781311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:02:59.781311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:02:59.781311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:02:59.781311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:02:59.781311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:02:59.781311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:02:59.781311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:02:59.781311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:02:59.781311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:02:59.781311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 12:02:59.781311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 12:02:59.781311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 12:02:59.782297 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 12:02:59.782297 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:02:59.782297 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:02:59.782297 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:02:59.782297 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:02:59.782297 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:02:59.782297 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:02:59.782297 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:02:59.782297 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:02:59.782297 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:02:59.782297 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:02:59.782297 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:02:59.782297 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:02:59.782297 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:02:59.782297 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:02:59.782297 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:02:59.782297 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:02:59.783226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 12:02:59.783226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 12:02:59.783226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:02:59.783226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:02:59.783226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:02:59.783226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:02:59.783226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:02:59.783226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:02:59.783226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 534/1090] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:02:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:02:59.789081 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed72395-1131-4a36-a13b-23e79ec17dc0 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:59.795961 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 557/1091] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:02:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 12:02:59.909693 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed72395-1131-4a36-a13b-23e79ec17dc0 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:02:59.919293 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:02:59.948244 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546ba47-0278-43cd-ab52-3e0e23aeac83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:02:59.948859 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546ba47-0278-43cd-ab52-3e0e23aeac83 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:02:59.949146 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546ba47-0278-43cd-ab52-3e0e23aeac83 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:02:59.949420 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546ba47-0278-43cd-ab52-3e0e23aeac83 None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:02:59.950112 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7546ba47-0278-43cd-ab52-3e0e23aeac83 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:02:59.950426 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7546ba47-0278-43cd-ab52-3e0e23aeac83 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:02:59.951925 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7546ba47-0278-43cd-ab52-3e0e23aeac83 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:02:59.965146 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 535/1092] 10.222.0.98 () {56 vars in 1148 bytes} [Thu Aug 18 12:02:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:02:59.971084 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d26f69-2059-4f06-9f2d-8015be8a5ee8 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:02:59.986631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:00.062316 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546ba47-0278-43cd-ab52-3e0e23aeac83 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:00.076236 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d26f69-2059-4f06-9f2d-8015be8a5ee8 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:00.111650 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33268cac-680f-490e-9270-15351b8dfbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:00.112175 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33268cac-680f-490e-9270-15351b8dfbac None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:00.112465 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33268cac-680f-490e-9270-15351b8dfbac None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:00.112741 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33268cac-680f-490e-9270-15351b8dfbac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:00.113473 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33268cac-680f-490e-9270-15351b8dfbac None admin] RBAC: Adding query filter params (role.id=9bcab8f185d94480b7fb843825dfb542, scope.project.id=6f676a5169464d83af4267db25025728, user.id=171c72a6b29e418bbb4ea5c6faffe3b3) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:03:00.113814 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33268cac-680f-490e-9270-15351b8dfbac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:00.115250 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33268cac-680f-490e-9270-15351b8dfbac None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:00.121059 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 558/1093] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:02:59 2022] GET /identity/v3/role_assignments?user.id=171c72a6b29e418bbb4ea5c6faffe3b3&scope.project.id=6f676a5169464d83af4267db25025728&role.id=9bcab8f185d94480b7fb843825dfb542 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:00.169611 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546ba47-0278-43cd-ab52-3e0e23aeac83 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:00.206207 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc860d5-52b5-45bc-b0ac-a521ec436b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:00.206845 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc860d5-52b5-45bc-b0ac-a521ec436b87 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:00.207129 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc860d5-52b5-45bc-b0ac-a521ec436b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:00.208065 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc860d5-52b5-45bc-b0ac-a521ec436b87 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:00.219302 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc860d5-52b5-45bc-b0ac-a521ec436b87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:03:00.224877 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-bcc860d5-52b5-45bc-b0ac-a521ec436b87 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:03:00.224877 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:03:00.224877 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:03:00.224877 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:03:00.224877 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:03:00.224877 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:03:00.224877 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:03:00.224877 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:03:00.224877 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:03:00.224877 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:03:00.224877 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:03:00.224877 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:03:00.224877 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:03:00.224877 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:03:00.224877 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:03:00.224877 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:03:00.225988 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:03:00.225988 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:03:00.225988 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:03:00.225988 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:03:00.225988 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:03:00.225988 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:03:00.225988 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:03:00.225988 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:03:00.225988 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:03:00.225988 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:03:00.225988 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:03:00.225988 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:03:00.225988 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:03:00.225988 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:03:00.225988 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:03:00.225988 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:03:00.226870 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:03:00.226870 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:03:00.226870 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:03:00.226870 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:03:00.226870 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:03:00.226870 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:03:00.226870 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:03:00.226870 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:03:00.226870 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:03:00.226870 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:03:00.226870 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:03:00.226870 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:03:00.226870 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 536/1094] 10.222.0.98 () {56 vars in 1096 bytes} [Thu Aug 18 12:03:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:03:00.232455 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33268cac-680f-490e-9270-15351b8dfbac None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:00.310967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:00.336562 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33268cac-680f-490e-9270-15351b8dfbac None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:00.372555 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489da9c7-dee2-40ae-bf9c-dba81b930d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:00.373171 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489da9c7-dee2-40ae-bf9c-dba81b930d20 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:00.373461 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489da9c7-dee2-40ae-bf9c-dba81b930d20 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:00.373743 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489da9c7-dee2-40ae-bf9c-dba81b930d20 None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:00.374412 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489da9c7-dee2-40ae-bf9c-dba81b930d20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:03:00.374722 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489da9c7-dee2-40ae-bf9c-dba81b930d20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:00.376323 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489da9c7-dee2-40ae-bf9c-dba81b930d20 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:00.386853 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 559/1095] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:03:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:00.393096 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc860d5-52b5-45bc-b0ac-a521ec436b87 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:00.423439 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:00.501036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc860d5-52b5-45bc-b0ac-a521ec436b87 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:00.537763 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f25f58-2940-4978-80ad-0b86eb32cba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:00.538398 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f25f58-2940-4978-80ad-0b86eb32cba7 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:00.538688 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f25f58-2940-4978-80ad-0b86eb32cba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:00.538968 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f25f58-2940-4978-80ad-0b86eb32cba7 None admin] PATH_INFO: `/v3/users/830f8a6395f9481c84654492406bb3bc` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:00.540171 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f25f58-2940-4978-80ad-0b86eb32cba7 None admin] RBAC: Authorizing `identity:get_user(user_id=830f8a6395f9481c84654492406bb3bc)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:00.541630 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f25f58-2940-4978-80ad-0b86eb32cba7 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:00.543217 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 537/1096] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:03:00 2022] GET /identity/v3/users/830f8a6395f9481c84654492406bb3bc => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:00.549351 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489da9c7-dee2-40ae-bf9c-dba81b930d20 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:00.659537 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489da9c7-dee2-40ae-bf9c-dba81b930d20 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:00.701270 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd2bf3-d796-46a3-81d1-bab9c4574b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:00.701907 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd2bf3-d796-46a3-81d1-bab9c4574b82 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:00.702190 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd2bf3-d796-46a3-81d1-bab9c4574b82 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:00.702464 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd2bf3-d796-46a3-81d1-bab9c4574b82 None admin] PATH_INFO: `/v3/projects/fafb788ca7ac448e903522b57eb1eb5e` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:00.703634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd2bf3-d796-46a3-81d1-bab9c4574b82 None admin] RBAC: Authorizing `identity:get_project(project_id=fafb788ca7ac448e903522b57eb1eb5e)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:00.705075 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd2bf3-d796-46a3-81d1-bab9c4574b82 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:00.706708 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 560/1097] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:03:00 2022] GET /identity/v3/projects/fafb788ca7ac448e903522b57eb1eb5e => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:00.712960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f25f58-2940-4978-80ad-0b86eb32cba7 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:00.814569 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:00.821793 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f25f58-2940-4978-80ad-0b86eb32cba7 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:00.856533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c3c6eb-60bf-4c3e-bab8-c2942cf989a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:00.857203 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3c6eb-60bf-4c3e-bab8-c2942cf989a7 None admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:00.857486 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3c6eb-60bf-4c3e-bab8-c2942cf989a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:00.857762 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3c6eb-60bf-4c3e-bab8-c2942cf989a7 None admin] PATH_INFO: `/v3/projects/fafb788ca7ac448e903522b57eb1eb5e/users/830f8a6395f9481c84654492406bb3bc/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:00.863063 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c3c6eb-60bf-4c3e-bab8-c2942cf989a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=fafb788ca7ac448e903522b57eb1eb5e, user_id=830f8a6395f9481c84654492406bb3bc, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:00.865132 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c3c6eb-60bf-4c3e-bab8-c2942cf989a7 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:00.937336 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 538/1098] 10.222.0.98 () {58 vars in 1439 bytes} [Thu Aug 18 12:03:00 2022] PUT /identity/v3/projects/fafb788ca7ac448e903522b57eb1eb5e/users/830f8a6395f9481c84654492406bb3bc/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:03:02.819558 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a37d45-ea63-41cc-830a-fa55319f3e40 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:02.820104 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a37d45-ea63-41cc-830a-fa55319f3e40 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:02.820104 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a37d45-ea63-41cc-830a-fa55319f3e40 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:02.821428 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 561/1099] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:03:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:03:02.826534 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e979ce-ea46-427d-8189-24c1328eeda1 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:02.826907 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e979ce-ea46-427d-8189-24c1328eeda1 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:02.827096 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e979ce-ea46-427d-8189-24c1328eeda1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:02.845668 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-f0e979ce-ea46-427d-8189-24c1328eeda1 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:03:02.846861 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-f0e979ce-ea46-427d-8189-24c1328eeda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:03:02.911479 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 539/1100] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:03:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:03:03.180346 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a37d45-ea63-41cc-830a-fa55319f3e40 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:03.184246 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:03.279961 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a37d45-ea63-41cc-830a-fa55319f3e40 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:03.313959 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8022c-a1f5-4300-8da5-9a2789b2e104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:03.314583 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8022c-a1f5-4300-8da5-9a2789b2e104 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:03.314864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8022c-a1f5-4300-8da5-9a2789b2e104 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:03.315135 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8022c-a1f5-4300-8da5-9a2789b2e104 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:03.325568 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af8022c-a1f5-4300-8da5-9a2789b2e104 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:03:03.330133 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-6af8022c-a1f5-4300-8da5-9a2789b2e104 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:03:03.330133 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:03:03.330133 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:03:03.330133 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:03:03.330133 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:03:03.330133 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:03:03.330133 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:03:03.330133 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:03:03.330133 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:03:03.330133 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:03:03.330133 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:03:03.330133 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:03:03.330133 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:03:03.330133 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:03:03.330133 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:03:03.330133 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:03:03.331235 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:03:03.331235 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:03:03.331235 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:03:03.331235 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:03:03.331235 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:03:03.331235 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:03:03.331235 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:03:03.331235 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:03:03.331235 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:03:03.331235 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:03:03.331235 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:03:03.331235 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:03:03.331235 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:03:03.331235 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:03:03.331235 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:03:03.331235 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:03:03.332154 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:03:03.332154 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:03:03.332154 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:03:03.332154 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:03:03.332154 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:03:03.332154 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:03:03.332154 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:03:03.332154 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:03:03.332154 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:03:03.332154 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:03:03.332154 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:03:03.332154 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:03:03.332154 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 562/1101] 10.222.0.98 () {56 vars in 1096 bytes} [Thu Aug 18 12:03:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:03:03.337315 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e979ce-ea46-427d-8189-24c1328eeda1 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:03.418697 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:03.434534 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e979ce-ea46-427d-8189-24c1328eeda1 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:03.467485 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6080bb85-a30a-41d4-9491-18c94e5e3c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:03.468094 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080bb85-a30a-41d4-9491-18c94e5e3c55 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:03.468453 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080bb85-a30a-41d4-9491-18c94e5e3c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:03.468730 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080bb85-a30a-41d4-9491-18c94e5e3c55 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:03.469387 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6080bb85-a30a-41d4-9491-18c94e5e3c55 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:03:03.469727 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6080bb85-a30a-41d4-9491-18c94e5e3c55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:03.471081 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6080bb85-a30a-41d4-9491-18c94e5e3c55 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:03.480837 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 540/1102] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:03:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:03.486704 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8022c-a1f5-4300-8da5-9a2789b2e104 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:03.550210 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:03.591280 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8022c-a1f5-4300-8da5-9a2789b2e104 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:03.625188 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0660bb-22fa-43f5-8119-367b8e62a76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:03.625806 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0660bb-22fa-43f5-8119-367b8e62a76c None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:03.626091 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0660bb-22fa-43f5-8119-367b8e62a76c None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:03.626370 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0660bb-22fa-43f5-8119-367b8e62a76c None admin] PATH_INFO: `/v3/users/830f8a6395f9481c84654492406bb3bc` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:03.627523 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0660bb-22fa-43f5-8119-367b8e62a76c None admin] RBAC: Authorizing `identity:get_user(user_id=830f8a6395f9481c84654492406bb3bc)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:03.628961 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0660bb-22fa-43f5-8119-367b8e62a76c None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:03.630561 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 563/1103] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:03:03 2022] GET /identity/v3/users/830f8a6395f9481c84654492406bb3bc => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:03.636529 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6080bb85-a30a-41d4-9491-18c94e5e3c55 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:03.735251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6080bb85-a30a-41d4-9491-18c94e5e3c55 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:03.767249 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b85b2-08a6-4455-8a96-2ea26785cbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:03.767881 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b85b2-08a6-4455-8a96-2ea26785cbd1 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:03.768168 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b85b2-08a6-4455-8a96-2ea26785cbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:03.768450 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b85b2-08a6-4455-8a96-2ea26785cbd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:03.783944 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:03.835072 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b85b2-08a6-4455-8a96-2ea26785cbd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:03.836560 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b85b2-08a6-4455-8a96-2ea26785cbd1 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:03.931647 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 541/1104] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:03:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 12:03:03.937741 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0660bb-22fa-43f5-8119-367b8e62a76c None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:04.080005 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0660bb-22fa-43f5-8119-367b8e62a76c None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:04.108178 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:04.112964 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313fb87b-6ea9-42df-af40-69c337e4945c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:04.113582 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313fb87b-6ea9-42df-af40-69c337e4945c None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:04.113862 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313fb87b-6ea9-42df-af40-69c337e4945c None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:04.119768 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313fb87b-6ea9-42df-af40-69c337e4945c None admin] PATH_INFO: `/v3/projects/fafb788ca7ac448e903522b57eb1eb5e` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:04.119768 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313fb87b-6ea9-42df-af40-69c337e4945c None admin] RBAC: Authorizing `identity:get_project(project_id=fafb788ca7ac448e903522b57eb1eb5e)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:04.119768 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313fb87b-6ea9-42df-af40-69c337e4945c None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:04.119768 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 564/1105] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:03:03 2022] GET /identity/v3/projects/fafb788ca7ac448e903522b57eb1eb5e => generated 311 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:04.124662 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b85b2-08a6-4455-8a96-2ea26785cbd1 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:04.171179 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:04.222353 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b85b2-08a6-4455-8a96-2ea26785cbd1 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:04.256627 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535cd4a-d452-4b66-931f-4dfb00741cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:04.257229 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535cd4a-d452-4b66-931f-4dfb00741cd1 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:04.257515 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535cd4a-d452-4b66-931f-4dfb00741cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:04.257792 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535cd4a-d452-4b66-931f-4dfb00741cd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:04.258515 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f535cd4a-d452-4b66-931f-4dfb00741cd1 None admin] RBAC: Adding query filter params (role.id=c53a0e13b58642f291c856efe1f0b87d, scope.project.id=fafb788ca7ac448e903522b57eb1eb5e, user.id=830f8a6395f9481c84654492406bb3bc) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:03:04.258828 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f535cd4a-d452-4b66-931f-4dfb00741cd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:04.260272 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f535cd4a-d452-4b66-931f-4dfb00741cd1 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:04.265880 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 542/1106] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:03:04 2022] GET /identity/v3/role_assignments?user.id=830f8a6395f9481c84654492406bb3bc&scope.project.id=fafb788ca7ac448e903522b57eb1eb5e&role.id=c53a0e13b58642f291c856efe1f0b87d => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:06.199043 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d67cf-22fc-4506-998b-60cc4f148393 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:06.199666 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d67cf-22fc-4506-998b-60cc4f148393 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:06.199772 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d67cf-22fc-4506-998b-60cc4f148393 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:06.201072 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 565/1107] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:03:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:03:06.206499 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d314d-e4d9-42e0-bf72-fe03751d7041 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:06.207014 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d314d-e4d9-42e0-bf72-fe03751d7041 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:06.207116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d314d-e4d9-42e0-bf72-fe03751d7041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:06.227393 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-1e1d314d-e4d9-42e0-bf72-fe03751d7041 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:03:06.228628 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-1e1d314d-e4d9-42e0-bf72-fe03751d7041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:03:06.305169 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 543/1108] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:03:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:03:06.576353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d67cf-22fc-4506-998b-60cc4f148393 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:06.679653 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d67cf-22fc-4506-998b-60cc4f148393 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:06.716618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f04597-59d8-4b99-8f06-0fefc468f3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:06.717241 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f04597-59d8-4b99-8f06-0fefc468f3ad None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:06.717523 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f04597-59d8-4b99-8f06-0fefc468f3ad None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:06.717797 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f04597-59d8-4b99-8f06-0fefc468f3ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:06.718896 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f04597-59d8-4b99-8f06-0fefc468f3ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:06.720353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f04597-59d8-4b99-8f06-0fefc468f3ad None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:06.721900 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 566/1109] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:03:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:06.727747 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d314d-e4d9-42e0-bf72-fe03751d7041 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:06.767709 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:06.823878 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d314d-e4d9-42e0-bf72-fe03751d7041 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:06.856480 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76e97a-d7db-4824-9001-b3eb92e17a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:06.857021 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76e97a-d7db-4824-9001-b3eb92e17a0c None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:06.857304 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76e97a-d7db-4824-9001-b3eb92e17a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:06.857578 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76e97a-d7db-4824-9001-b3eb92e17a0c None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:06.858233 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76e97a-d7db-4824-9001-b3eb92e17a0c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:06.859732 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76e97a-d7db-4824-9001-b3eb92e17a0c None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:06.971250 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 544/1110] 10.222.0.98 () {60 vars in 1132 bytes} [Thu Aug 18 12:03:06 2022] POST /identity/v3/users => generated 286 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:03:08.954662 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd593325-d048-4705-926e-fca807de3e17 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:08.955245 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd593325-d048-4705-926e-fca807de3e17 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:08.955245 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd593325-d048-4705-926e-fca807de3e17 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:08.956563 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 567/1111] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:03:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:03:08.962321 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748b552-1ac6-4c8b-8720-1e65fa00ebfa None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:08.962765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748b552-1ac6-4c8b-8720-1e65fa00ebfa None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:08.962885 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748b552-1ac6-4c8b-8720-1e65fa00ebfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:08.981295 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-d748b552-1ac6-4c8b-8720-1e65fa00ebfa None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:03:08.982500 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-d748b552-1ac6-4c8b-8720-1e65fa00ebfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:03:09.048584 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 545/1112] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:03:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 12:03:09.311297 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd593325-d048-4705-926e-fca807de3e17 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:09.381542 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:09.415056 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd593325-d048-4705-926e-fca807de3e17 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:09.448285 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a827fb8a-ea20-4042-9315-c09db13a4bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:09.448927 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a827fb8a-ea20-4042-9315-c09db13a4bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:09.449204 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a827fb8a-ea20-4042-9315-c09db13a4bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:09.449506 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a827fb8a-ea20-4042-9315-c09db13a4bf6 None admin] PATH_INFO: `/v3/roles/3a4b7e53b8644e508ac440f71138be16` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:09.450878 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a827fb8a-ea20-4042-9315-c09db13a4bf6 None admin] RBAC: Authorizing `identity:get_role(role_id=3a4b7e53b8644e508ac440f71138be16)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:09.452285 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a827fb8a-ea20-4042-9315-c09db13a4bf6 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:09.453603 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 568/1113] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:03:09 2022] GET /identity/v3/roles/3a4b7e53b8644e508ac440f71138be16 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:09.460058 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d748b552-1ac6-4c8b-8720-1e65fa00ebfa None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:09.486000 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:09.557449 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d748b552-1ac6-4c8b-8720-1e65fa00ebfa None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:09.591395 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6587a3a5-e239-4c1e-a04e-a8648ce5c620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:09.591990 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587a3a5-e239-4c1e-a04e-a8648ce5c620 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:09.592278 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587a3a5-e239-4c1e-a04e-a8648ce5c620 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:09.592549 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587a3a5-e239-4c1e-a04e-a8648ce5c620 None admin] PATH_INFO: `/v3/users/11324d32cccf4f9baaa6897233b0a979` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:09.605543 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6587a3a5-e239-4c1e-a04e-a8648ce5c620 None admin] RBAC: Authorizing `identity:get_user(user_id=11324d32cccf4f9baaa6897233b0a979)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:09.609765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6587a3a5-e239-4c1e-a04e-a8648ce5c620 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:09.609765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 546/1114] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:03:09 2022] GET /identity/v3/users/11324d32cccf4f9baaa6897233b0a979 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:03:09.614531 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a827fb8a-ea20-4042-9315-c09db13a4bf6 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:09.718896 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a827fb8a-ea20-4042-9315-c09db13a4bf6 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:09.752132 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f51895-a26d-4f9d-8aa4-30e454b5c57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:09.752740 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f51895-a26d-4f9d-8aa4-30e454b5c57d None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:09.753028 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f51895-a26d-4f9d-8aa4-30e454b5c57d None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:09.753308 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f51895-a26d-4f9d-8aa4-30e454b5c57d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:09.756694 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:09.818657 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f51895-a26d-4f9d-8aa4-30e454b5c57d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:09.820120 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f51895-a26d-4f9d-8aa4-30e454b5c57d None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:09.914501 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 569/1115] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:03:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 12:03:09.920469 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6587a3a5-e239-4c1e-a04e-a8648ce5c620 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:10.006430 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:10.022267 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6587a3a5-e239-4c1e-a04e-a8648ce5c620 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:10.056555 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea7699-4ee7-4396-ae8e-38917fcbfab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:10.057180 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea7699-4ee7-4396-ae8e-38917fcbfab4 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:10.057465 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea7699-4ee7-4396-ae8e-38917fcbfab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:10.057743 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea7699-4ee7-4396-ae8e-38917fcbfab4 None admin] PATH_INFO: `/v3/projects/fafb788ca7ac448e903522b57eb1eb5e` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:10.058865 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ea7699-4ee7-4396-ae8e-38917fcbfab4 None admin] RBAC: Authorizing `identity:get_project(project_id=fafb788ca7ac448e903522b57eb1eb5e)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:10.060325 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ea7699-4ee7-4396-ae8e-38917fcbfab4 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:10.061959 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 547/1116] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:03:09 2022] GET /identity/v3/projects/fafb788ca7ac448e903522b57eb1eb5e => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:10.068886 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f51895-a26d-4f9d-8aa4-30e454b5c57d None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:10.102651 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:10.168629 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f51895-a26d-4f9d-8aa4-30e454b5c57d None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:10.201207 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb2fcf-2b69-4c9b-9926-35b8fda274c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:10.201792 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb2fcf-2b69-4c9b-9926-35b8fda274c5 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:10.202072 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb2fcf-2b69-4c9b-9926-35b8fda274c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:10.202344 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb2fcf-2b69-4c9b-9926-35b8fda274c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:10.203056 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fb2fcf-2b69-4c9b-9926-35b8fda274c5 None admin] RBAC: Adding query filter params (role.id=3a4b7e53b8644e508ac440f71138be16, scope.project.id=fafb788ca7ac448e903522b57eb1eb5e, user.id=11324d32cccf4f9baaa6897233b0a979) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:03:10.203364 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fb2fcf-2b69-4c9b-9926-35b8fda274c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:10.204806 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fb2fcf-2b69-4c9b-9926-35b8fda274c5 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:10.209967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 570/1117] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:03:10 2022] GET /identity/v3/role_assignments?user.id=11324d32cccf4f9baaa6897233b0a979&scope.project.id=fafb788ca7ac448e903522b57eb1eb5e&role.id=3a4b7e53b8644e508ac440f71138be16 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:12.126282 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdb894-5d24-4c75-aff6-4780fb718e4e None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:12.126915 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdb894-5d24-4c75-aff6-4780fb718e4e None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:12.126915 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdb894-5d24-4c75-aff6-4780fb718e4e None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:12.128198 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 548/1118] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:03:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:03:12.134037 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8feb88-a7a8-40fc-8072-07b9dc131a2c None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:12.134470 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8feb88-a7a8-40fc-8072-07b9dc131a2c None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:12.134623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8feb88-a7a8-40fc-8072-07b9dc131a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:12.152274 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-9c8feb88-a7a8-40fc-8072-07b9dc131a2c None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:03:12.153458 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-9c8feb88-a7a8-40fc-8072-07b9dc131a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:03:12.218230 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 571/1119] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:03:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:03:12.484125 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcdb894-5d24-4c75-aff6-4780fb718e4e None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:12.585350 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcdb894-5d24-4c75-aff6-4780fb718e4e None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:12.624651 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567709e-d44f-4d93-b7af-1da41dd849bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:12.625286 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567709e-d44f-4d93-b7af-1da41dd849bf None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:12.625572 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567709e-d44f-4d93-b7af-1da41dd849bf None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:12.625848 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567709e-d44f-4d93-b7af-1da41dd849bf None admin] PATH_INFO: `/v3/roles/3a4b7e53b8644e508ac440f71138be16` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:12.627280 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2567709e-d44f-4d93-b7af-1da41dd849bf None admin] RBAC: Authorizing `identity:get_role(role_id=3a4b7e53b8644e508ac440f71138be16)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:12.628685 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2567709e-d44f-4d93-b7af-1da41dd849bf None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:12.630012 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 549/1120] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:03:12 2022] GET /identity/v3/roles/3a4b7e53b8644e508ac440f71138be16 => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:12.636089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8feb88-a7a8-40fc-8072-07b9dc131a2c None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:12.666291 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:12.739936 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8feb88-a7a8-40fc-8072-07b9dc131a2c None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:12.774591 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f030667-c5df-497a-a6b7-c90ed82c2297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:12.775218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f030667-c5df-497a-a6b7-c90ed82c2297 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:12.775527 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f030667-c5df-497a-a6b7-c90ed82c2297 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:12.775805 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f030667-c5df-497a-a6b7-c90ed82c2297 None admin] PATH_INFO: `/v3/users/11324d32cccf4f9baaa6897233b0a979` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:12.776923 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f030667-c5df-497a-a6b7-c90ed82c2297 None admin] RBAC: Authorizing `identity:get_user(user_id=11324d32cccf4f9baaa6897233b0a979)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:12.778352 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f030667-c5df-497a-a6b7-c90ed82c2297 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:12.780225 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 572/1121] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:03:12 2022] GET /identity/v3/users/11324d32cccf4f9baaa6897233b0a979 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:12.786030 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567709e-d44f-4d93-b7af-1da41dd849bf None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:12.808148 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:12.883434 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567709e-d44f-4d93-b7af-1da41dd849bf None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:12.917971 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a66836c-cc67-49fb-adee-c784e61fa52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:12.918592 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66836c-cc67-49fb-adee-c784e61fa52d None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:12.918875 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66836c-cc67-49fb-adee-c784e61fa52d None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:12.919150 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66836c-cc67-49fb-adee-c784e61fa52d None admin] PATH_INFO: `/v3/projects/fafb788ca7ac448e903522b57eb1eb5e` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:12.920288 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a66836c-cc67-49fb-adee-c784e61fa52d None admin] RBAC: Authorizing `identity:get_project(project_id=fafb788ca7ac448e903522b57eb1eb5e)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:12.921710 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a66836c-cc67-49fb-adee-c784e61fa52d None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:12.923329 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 550/1122] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:03:12 2022] GET /identity/v3/projects/fafb788ca7ac448e903522b57eb1eb5e => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:12.929402 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f030667-c5df-497a-a6b7-c90ed82c2297 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:13.030316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:13.034330 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f030667-c5df-497a-a6b7-c90ed82c2297 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:13.072988 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a373360-5d9f-445d-a98d-8a2f368262d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:13.073650 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a373360-5d9f-445d-a98d-8a2f368262d0 None admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:13.073932 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a373360-5d9f-445d-a98d-8a2f368262d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:13.074211 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a373360-5d9f-445d-a98d-8a2f368262d0 None admin] PATH_INFO: `/v3/projects/fafb788ca7ac448e903522b57eb1eb5e/users/11324d32cccf4f9baaa6897233b0a979/roles/3a4b7e53b8644e508ac440f71138be16` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:13.076384 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a373360-5d9f-445d-a98d-8a2f368262d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=fafb788ca7ac448e903522b57eb1eb5e, user_id=11324d32cccf4f9baaa6897233b0a979, role_id=3a4b7e53b8644e508ac440f71138be16)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:13.077821 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a373360-5d9f-445d-a98d-8a2f368262d0 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:13.126076 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 573/1123] 10.222.0.98 () {58 vars in 1439 bytes} [Thu Aug 18 12:03:12 2022] PUT /identity/v3/projects/fafb788ca7ac448e903522b57eb1eb5e/users/11324d32cccf4f9baaa6897233b0a979/roles/3a4b7e53b8644e508ac440f71138be16 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:03:15.000615 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cad59-d732-4521-87db-4c2ddc492dc0 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:15.001148 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cad59-d732-4521-87db-4c2ddc492dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:15.001148 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cad59-d732-4521-87db-4c2ddc492dc0 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:15.002497 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 551/1124] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:03:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:03:15.008225 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2565781-9d9b-40ef-85e9-c53c8b7fd2e9 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:15.008687 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2565781-9d9b-40ef-85e9-c53c8b7fd2e9 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:15.008824 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2565781-9d9b-40ef-85e9-c53c8b7fd2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:15.026356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-f2565781-9d9b-40ef-85e9-c53c8b7fd2e9 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:03:15.027575 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-f2565781-9d9b-40ef-85e9-c53c8b7fd2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:03:15.092186 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 574/1125] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:03:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:03:15.358922 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5cad59-d732-4521-87db-4c2ddc492dc0 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:15.446399 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:15.459583 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5cad59-d732-4521-87db-4c2ddc492dc0 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:15.492212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2504d-a8b9-492e-b9f3-52b000abade5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:15.492836 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2504d-a8b9-492e-b9f3-52b000abade5 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:15.493119 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2504d-a8b9-492e-b9f3-52b000abade5 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:15.493396 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2504d-a8b9-492e-b9f3-52b000abade5 None admin] PATH_INFO: `/v3/roles/3a4b7e53b8644e508ac440f71138be16` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:15.494787 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e2504d-a8b9-492e-b9f3-52b000abade5 None admin] RBAC: Authorizing `identity:get_role(role_id=3a4b7e53b8644e508ac440f71138be16)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:15.496178 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e2504d-a8b9-492e-b9f3-52b000abade5 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:15.497488 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 552/1126] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:03:15 2022] GET /identity/v3/roles/3a4b7e53b8644e508ac440f71138be16 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:03:15.503350 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2565781-9d9b-40ef-85e9-c53c8b7fd2e9 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:15.587844 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:15.599366 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2565781-9d9b-40ef-85e9-c53c8b7fd2e9 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:15.636334 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc126fe-f2d8-435f-8e29-c4e0cf8a4a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:15.636334 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc126fe-f2d8-435f-8e29-c4e0cf8a4a00 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:15.636334 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc126fe-f2d8-435f-8e29-c4e0cf8a4a00 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:15.636334 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc126fe-f2d8-435f-8e29-c4e0cf8a4a00 None admin] PATH_INFO: `/v3/users/11324d32cccf4f9baaa6897233b0a979` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:15.636729 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc126fe-f2d8-435f-8e29-c4e0cf8a4a00 None admin] RBAC: Authorizing `identity:get_user(user_id=11324d32cccf4f9baaa6897233b0a979)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:15.636905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc126fe-f2d8-435f-8e29-c4e0cf8a4a00 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:15.638468 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 575/1127] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:03:15 2022] GET /identity/v3/users/11324d32cccf4f9baaa6897233b0a979 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:15.644398 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2504d-a8b9-492e-b9f3-52b000abade5 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:15.743305 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2504d-a8b9-492e-b9f3-52b000abade5 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:15.776411 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9608b7a5-3285-4205-9230-60929133f331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:15.777013 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608b7a5-3285-4205-9230-60929133f331 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:15.777298 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608b7a5-3285-4205-9230-60929133f331 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:15.777571 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608b7a5-3285-4205-9230-60929133f331 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:15.801328 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:15.843133 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9608b7a5-3285-4205-9230-60929133f331 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:15.844618 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9608b7a5-3285-4205-9230-60929133f331 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:15.939275 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 553/1128] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:03:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 18 12:03:15.945180 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc126fe-f2d8-435f-8e29-c4e0cf8a4a00 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:16.042961 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc126fe-f2d8-435f-8e29-c4e0cf8a4a00 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:16.075732 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6025ee-c2c6-44bd-9199-1d5013e83bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:16.076350 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6025ee-c2c6-44bd-9199-1d5013e83bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:16.076624 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6025ee-c2c6-44bd-9199-1d5013e83bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:16.076896 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6025ee-c2c6-44bd-9199-1d5013e83bd0 None admin] PATH_INFO: `/v3/projects/fafb788ca7ac448e903522b57eb1eb5e` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:16.078000 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6025ee-c2c6-44bd-9199-1d5013e83bd0 None admin] RBAC: Authorizing `identity:get_project(project_id=fafb788ca7ac448e903522b57eb1eb5e)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:16.079445 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6025ee-c2c6-44bd-9199-1d5013e83bd0 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:16.081040 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 576/1129] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:03:15 2022] GET /identity/v3/projects/fafb788ca7ac448e903522b57eb1eb5e => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:16.087990 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9608b7a5-3285-4205-9230-60929133f331 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:16.143115 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:16.184795 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9608b7a5-3285-4205-9230-60929133f331 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:16.219351 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664f338f-ec67-4953-b6a0-a38946dad348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:16.219990 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f338f-ec67-4953-b6a0-a38946dad348 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:16.220274 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f338f-ec67-4953-b6a0-a38946dad348 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:16.220555 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f338f-ec67-4953-b6a0-a38946dad348 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:16.221274 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664f338f-ec67-4953-b6a0-a38946dad348 None admin] RBAC: Adding query filter params (role.id=3a4b7e53b8644e508ac440f71138be16, scope.project.id=fafb788ca7ac448e903522b57eb1eb5e, user.id=11324d32cccf4f9baaa6897233b0a979) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:03:16.221582 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664f338f-ec67-4953-b6a0-a38946dad348 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:16.223016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664f338f-ec67-4953-b6a0-a38946dad348 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:16.229183 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 554/1130] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:03:16 2022] GET /identity/v3/role_assignments?user.id=11324d32cccf4f9baaa6897233b0a979&scope.project.id=fafb788ca7ac448e903522b57eb1eb5e&role.id=3a4b7e53b8644e508ac440f71138be16 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:18.140234 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeda418-3da1-4249-a3f0-62b8b20de86c None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:18.140800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeda418-3da1-4249-a3f0-62b8b20de86c None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:18.140800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeda418-3da1-4249-a3f0-62b8b20de86c None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:18.142234 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 577/1131] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:03:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:03:18.147514 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea543a7-2676-48bd-b75e-ec0c0bebbc7c None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:18.147934 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea543a7-2676-48bd-b75e-ec0c0bebbc7c None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:18.148078 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea543a7-2676-48bd-b75e-ec0c0bebbc7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:18.166267 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-6ea543a7-2676-48bd-b75e-ec0c0bebbc7c None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:03:18.167471 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-6ea543a7-2676-48bd-b75e-ec0c0bebbc7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:03:18.231354 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 555/1132] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:03:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:03:18.499015 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeda418-3da1-4249-a3f0-62b8b20de86c None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:18.524453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:18.599032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeda418-3da1-4249-a3f0-62b8b20de86c None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:18.633165 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9cf071-5722-4602-a767-84da494c34f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:18.633796 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cf071-5722-4602-a767-84da494c34f2 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:18.634078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cf071-5722-4602-a767-84da494c34f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:18.634352 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cf071-5722-4602-a767-84da494c34f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:18.635485 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9cf071-5722-4602-a767-84da494c34f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:18.636914 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9cf071-5722-4602-a767-84da494c34f2 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:18.638485 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 578/1133] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:03:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:18.644535 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea543a7-2676-48bd-b75e-ec0c0bebbc7c None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:18.696343 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:18.749292 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea543a7-2676-48bd-b75e-ec0c0bebbc7c None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:18.783463 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a4afcd-cd03-41fc-b8f9-59e923aaa204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:18.784075 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a4afcd-cd03-41fc-b8f9-59e923aaa204 None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:18.784361 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a4afcd-cd03-41fc-b8f9-59e923aaa204 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:18.784639 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a4afcd-cd03-41fc-b8f9-59e923aaa204 None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:18.785301 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a4afcd-cd03-41fc-b8f9-59e923aaa204 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:18.786725 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a4afcd-cd03-41fc-b8f9-59e923aaa204 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:18.871844 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 556/1134] 10.222.0.98 () {60 vars in 1141 bytes} [Thu Aug 18 12:03:18 2022] POST /identity/v3/projects => generated 311 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:03:20.819975 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b572e2c-01f3-4d93-94d1-17c0e70355dc None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:20.820503 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b572e2c-01f3-4d93-94d1-17c0e70355dc None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:20.820503 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b572e2c-01f3-4d93-94d1-17c0e70355dc None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:20.821840 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 579/1135] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:03:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:03:20.827082 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a300f-efdd-4abb-add2-fe80bf4a5a69 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:20.827542 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a300f-efdd-4abb-add2-fe80bf4a5a69 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:20.827685 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a300f-efdd-4abb-add2-fe80bf4a5a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:20.845441 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-614a300f-efdd-4abb-add2-fe80bf4a5a69 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:03:20.846637 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-614a300f-efdd-4abb-add2-fe80bf4a5a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:03:20.911763 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 557/1136] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:03:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:03:21.183947 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b572e2c-01f3-4d93-94d1-17c0e70355dc None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:21.274015 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:21.283703 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b572e2c-01f3-4d93-94d1-17c0e70355dc None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:21.319756 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57948c9e-c1df-4ba4-b281-a976f1e87fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:21.320386 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57948c9e-c1df-4ba4-b281-a976f1e87fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:21.320668 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57948c9e-c1df-4ba4-b281-a976f1e87fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:21.320948 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57948c9e-c1df-4ba4-b281-a976f1e87fc5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:21.322058 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57948c9e-c1df-4ba4-b281-a976f1e87fc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:21.323514 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57948c9e-c1df-4ba4-b281-a976f1e87fc5 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:21.325078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 580/1137] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:03:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:21.330979 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a300f-efdd-4abb-add2-fe80bf4a5a69 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:21.361269 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:21.428869 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a300f-efdd-4abb-add2-fe80bf4a5a69 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:21.462013 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38914b2-3d3f-47e4-a7a3-f7a8892fb814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:21.462608 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38914b2-3d3f-47e4-a7a3-f7a8892fb814 None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:21.462890 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38914b2-3d3f-47e4-a7a3-f7a8892fb814 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:21.463164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38914b2-3d3f-47e4-a7a3-f7a8892fb814 None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:21.463845 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38914b2-3d3f-47e4-a7a3-f7a8892fb814 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:21.465264 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38914b2-3d3f-47e4-a7a3-f7a8892fb814 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:21.577883 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 558/1138] 10.222.0.98 () {60 vars in 1132 bytes} [Thu Aug 18 12:03:21 2022] POST /identity/v3/users => generated 286 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:03:23.500025 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de66e2-8838-4f67-9492-2fd309bc919f None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:23.500613 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de66e2-8838-4f67-9492-2fd309bc919f None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:23.500613 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de66e2-8838-4f67-9492-2fd309bc919f None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:23.501915 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 581/1139] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:03:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:03:23.507044 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997217a-16ac-46fa-81fc-ea8a052d4f93 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:23.507516 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997217a-16ac-46fa-81fc-ea8a052d4f93 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:23.507636 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997217a-16ac-46fa-81fc-ea8a052d4f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:23.526003 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-1997217a-16ac-46fa-81fc-ea8a052d4f93 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:03:23.527208 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-1997217a-16ac-46fa-81fc-ea8a052d4f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:03:23.593454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 559/1140] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:03:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:03:23.864786 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de66e2-8838-4f67-9492-2fd309bc919f None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:23.964451 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de66e2-8838-4f67-9492-2fd309bc919f None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:23.997815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e96ac6-994d-4987-b098-40d053cea1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:23.998438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e96ac6-994d-4987-b098-40d053cea1c7 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:23.998717 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e96ac6-994d-4987-b098-40d053cea1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:23.998990 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e96ac6-994d-4987-b098-40d053cea1c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:24.008658 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e96ac6-994d-4987-b098-40d053cea1c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:03:24.013302 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-d3e96ac6-994d-4987-b098-40d053cea1c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:03:24.013302 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:03:24.013302 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:03:24.013302 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:03:24.013302 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:03:24.013302 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:03:24.013302 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:03:24.013302 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:03:24.013302 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:03:24.013302 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:03:24.013302 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:03:24.013302 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:03:24.013302 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:03:24.013302 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:03:24.013302 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:03:24.013302 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:03:24.014580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:03:24.014580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:03:24.014580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:03:24.014580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:03:24.014580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:03:24.014580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:03:24.014580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:03:24.014580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:03:24.014580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:03:24.014580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:03:24.014580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:03:24.014580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:03:24.014580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:03:24.014580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:03:24.014580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:03:24.014580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:03:24.015769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:03:24.015769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:03:24.015769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:03:24.015769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:03:24.015769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:03:24.015769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:03:24.015769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:03:24.015769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:03:24.015769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:03:24.015769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:03:24.015769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:03:24.015769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:03:24.015769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 582/1141] 10.222.0.98 () {56 vars in 1096 bytes} [Thu Aug 18 12:03:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:03:24.020600 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1997217a-16ac-46fa-81fc-ea8a052d4f93 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:24.039755 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:24.118090 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1997217a-16ac-46fa-81fc-ea8a052d4f93 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:24.151140 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113aea2-8bb4-4c06-8218-48ed2c9120fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:24.151755 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113aea2-8bb4-4c06-8218-48ed2c9120fc None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:24.152035 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113aea2-8bb4-4c06-8218-48ed2c9120fc None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:24.152308 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113aea2-8bb4-4c06-8218-48ed2c9120fc None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:24.152964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0113aea2-8bb4-4c06-8218-48ed2c9120fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:03:24.153266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0113aea2-8bb4-4c06-8218-48ed2c9120fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:24.154620 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0113aea2-8bb4-4c06-8218-48ed2c9120fc None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:24.163326 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 560/1142] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:03:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:24.169195 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e96ac6-994d-4987-b098-40d053cea1c7 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:24.190987 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:24.266534 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e96ac6-994d-4987-b098-40d053cea1c7 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:24.300540 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7460b55a-d430-48a7-a97d-af0924cea271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:24.301155 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460b55a-d430-48a7-a97d-af0924cea271 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:24.301430 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460b55a-d430-48a7-a97d-af0924cea271 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:24.301705 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460b55a-d430-48a7-a97d-af0924cea271 None admin] PATH_INFO: `/v3/users/ad3dcaf8bebe42359e2e14de59b267f3` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:24.314382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7460b55a-d430-48a7-a97d-af0924cea271 None admin] RBAC: Authorizing `identity:get_user(user_id=ad3dcaf8bebe42359e2e14de59b267f3)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:24.315868 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7460b55a-d430-48a7-a97d-af0924cea271 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:24.317454 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 583/1143] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:03:24 2022] GET /identity/v3/users/ad3dcaf8bebe42359e2e14de59b267f3 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:24.324011 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113aea2-8bb4-4c06-8218-48ed2c9120fc None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:24.403880 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:24.422271 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113aea2-8bb4-4c06-8218-48ed2c9120fc None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:24.455181 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877d141-ca70-44ec-8b20-9b246c29fa2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:24.455815 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877d141-ca70-44ec-8b20-9b246c29fa2e None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:24.456101 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877d141-ca70-44ec-8b20-9b246c29fa2e None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:24.456379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877d141-ca70-44ec-8b20-9b246c29fa2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:24.521877 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877d141-ca70-44ec-8b20-9b246c29fa2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:24.523872 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877d141-ca70-44ec-8b20-9b246c29fa2e None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:24.600888 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:24.619620 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 561/1144] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:03:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 12:03:24.625733 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7460b55a-d430-48a7-a97d-af0924cea271 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:24.715024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:24.736538 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7460b55a-d430-48a7-a97d-af0924cea271 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:24.769852 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128524db-88aa-47f2-93f7-f100adf0b5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:24.770479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128524db-88aa-47f2-93f7-f100adf0b5d5 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:24.770760 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128524db-88aa-47f2-93f7-f100adf0b5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:24.771034 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128524db-88aa-47f2-93f7-f100adf0b5d5 None admin] PATH_INFO: `/v3/projects/684f7df2f5b041f2b020c307c13e437b` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:24.772754 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128524db-88aa-47f2-93f7-f100adf0b5d5 None admin] RBAC: Authorizing `identity:get_project(project_id=684f7df2f5b041f2b020c307c13e437b)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:24.774185 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128524db-88aa-47f2-93f7-f100adf0b5d5 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:24.775854 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 584/1145] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:03:24 2022] GET /identity/v3/projects/684f7df2f5b041f2b020c307c13e437b => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:24.782334 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877d141-ca70-44ec-8b20-9b246c29fa2e None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:24.881515 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877d141-ca70-44ec-8b20-9b246c29fa2e None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:24.917403 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e6abe-ea1c-4152-88d7-53f05ba7cdc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:24.918010 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e6abe-ea1c-4152-88d7-53f05ba7cdc0 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:24.918291 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e6abe-ea1c-4152-88d7-53f05ba7cdc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:24.918566 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e6abe-ea1c-4152-88d7-53f05ba7cdc0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:24.919294 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e6abe-ea1c-4152-88d7-53f05ba7cdc0 None admin] RBAC: Adding query filter params (role.id=c53a0e13b58642f291c856efe1f0b87d, scope.project.id=684f7df2f5b041f2b020c307c13e437b, user.id=ad3dcaf8bebe42359e2e14de59b267f3) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:03:24.919642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e6abe-ea1c-4152-88d7-53f05ba7cdc0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:24.921067 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e6abe-ea1c-4152-88d7-53f05ba7cdc0 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:24.926463 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 562/1146] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:03:24 2022] GET /identity/v3/role_assignments?user.id=ad3dcaf8bebe42359e2e14de59b267f3&scope.project.id=684f7df2f5b041f2b020c307c13e437b&role.id=c53a0e13b58642f291c856efe1f0b87d => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:26.822783 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47aed1-12cd-4235-b1ef-4ff775863bfa None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:26.823351 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47aed1-12cd-4235-b1ef-4ff775863bfa None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:26.823351 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47aed1-12cd-4235-b1ef-4ff775863bfa None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:26.824700 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 585/1147] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:03:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:03:26.830302 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ee834-a92b-4f2e-91fe-0e8c16af58c1 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:26.830753 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ee834-a92b-4f2e-91fe-0e8c16af58c1 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:26.830867 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ee834-a92b-4f2e-91fe-0e8c16af58c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:26.849537 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-254ee834-a92b-4f2e-91fe-0e8c16af58c1 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:03:26.850808 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-254ee834-a92b-4f2e-91fe-0e8c16af58c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:03:26.889659 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:26.917258 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 563/1148] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:03:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:03:27.189649 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d47aed1-12cd-4235-b1ef-4ff775863bfa None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:27.289226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d47aed1-12cd-4235-b1ef-4ff775863bfa None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:27.323012 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60fe6ea-98a8-42e5-8d2e-46942b4db2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:27.323656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fe6ea-98a8-42e5-8d2e-46942b4db2ad None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:27.323945 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fe6ea-98a8-42e5-8d2e-46942b4db2ad None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:27.324222 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fe6ea-98a8-42e5-8d2e-46942b4db2ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:27.334122 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60fe6ea-98a8-42e5-8d2e-46942b4db2ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:03:27.338790 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-c60fe6ea-98a8-42e5-8d2e-46942b4db2ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:03:27.338790 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:03:27.338790 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:03:27.338790 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:03:27.338790 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:03:27.338790 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:03:27.338790 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:03:27.338790 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:03:27.338790 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:03:27.338790 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:03:27.338790 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:03:27.338790 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:03:27.338790 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:03:27.338790 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:03:27.338790 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:03:27.338790 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:03:27.339922 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:03:27.339922 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:03:27.339922 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:03:27.339922 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:03:27.339922 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:03:27.339922 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:03:27.339922 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:03:27.339922 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:03:27.339922 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:03:27.339922 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:03:27.339922 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:03:27.339922 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:03:27.339922 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:03:27.339922 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:03:27.339922 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:03:27.339922 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:03:27.340770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:03:27.340770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:03:27.340770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:03:27.340770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:03:27.340770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:03:27.340770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:03:27.340770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:03:27.340770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:03:27.340770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:03:27.340770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:03:27.340770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:03:27.340770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:03:27.340770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 586/1149] 10.222.0.98 () {56 vars in 1096 bytes} [Thu Aug 18 12:03:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:03:27.345984 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254ee834-a92b-4f2e-91fe-0e8c16af58c1 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:27.448492 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254ee834-a92b-4f2e-91fe-0e8c16af58c1 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:27.481517 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec29a5-8897-4b5d-977c-cadfee5ba43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:27.482115 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec29a5-8897-4b5d-977c-cadfee5ba43a None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:27.482399 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec29a5-8897-4b5d-977c-cadfee5ba43a None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:27.482675 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec29a5-8897-4b5d-977c-cadfee5ba43a None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:27.483363 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ec29a5-8897-4b5d-977c-cadfee5ba43a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:03:27.483700 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ec29a5-8897-4b5d-977c-cadfee5ba43a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:27.485061 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ec29a5-8897-4b5d-977c-cadfee5ba43a None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:27.493467 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 564/1150] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:03:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:27.499391 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60fe6ea-98a8-42e5-8d2e-46942b4db2ad None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:27.508890 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:27.598803 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60fe6ea-98a8-42e5-8d2e-46942b4db2ad None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:27.632921 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c9b24-59f0-45a0-b356-cf3fe76a18b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:27.633542 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c9b24-59f0-45a0-b356-cf3fe76a18b5 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:27.633828 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c9b24-59f0-45a0-b356-cf3fe76a18b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:27.634107 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c9b24-59f0-45a0-b356-cf3fe76a18b5 None admin] PATH_INFO: `/v3/users/ad3dcaf8bebe42359e2e14de59b267f3` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:27.635235 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266c9b24-59f0-45a0-b356-cf3fe76a18b5 None admin] RBAC: Authorizing `identity:get_user(user_id=ad3dcaf8bebe42359e2e14de59b267f3)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:27.636699 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266c9b24-59f0-45a0-b356-cf3fe76a18b5 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:27.638270 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 587/1151] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:03:27 2022] GET /identity/v3/users/ad3dcaf8bebe42359e2e14de59b267f3 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:27.644190 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec29a5-8897-4b5d-977c-cadfee5ba43a None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:27.681239 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:27.743982 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec29a5-8897-4b5d-977c-cadfee5ba43a None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:27.776559 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22486e75-fc45-4cc3-8e12-2df65c743ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:27.777186 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22486e75-fc45-4cc3-8e12-2df65c743ef4 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:27.777471 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22486e75-fc45-4cc3-8e12-2df65c743ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:27.777747 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22486e75-fc45-4cc3-8e12-2df65c743ef4 None admin] PATH_INFO: `/v3/projects/684f7df2f5b041f2b020c307c13e437b` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:27.778865 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22486e75-fc45-4cc3-8e12-2df65c743ef4 None admin] RBAC: Authorizing `identity:get_project(project_id=684f7df2f5b041f2b020c307c13e437b)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:27.780327 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22486e75-fc45-4cc3-8e12-2df65c743ef4 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:27.782208 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 565/1152] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:03:27 2022] GET /identity/v3/projects/684f7df2f5b041f2b020c307c13e437b => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:27.788947 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c9b24-59f0-45a0-b356-cf3fe76a18b5 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:27.862401 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:27.887339 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c9b24-59f0-45a0-b356-cf3fe76a18b5 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:27.921970 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1789633f-992a-46d1-86f0-499fe8b02868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:27.922622 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789633f-992a-46d1-86f0-499fe8b02868 None admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:27.922908 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789633f-992a-46d1-86f0-499fe8b02868 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:27.923188 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789633f-992a-46d1-86f0-499fe8b02868 None admin] PATH_INFO: `/v3/projects/684f7df2f5b041f2b020c307c13e437b/users/ad3dcaf8bebe42359e2e14de59b267f3/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:27.925111 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1789633f-992a-46d1-86f0-499fe8b02868 None admin] RBAC: Authorizing `identity:create_grant(project_id=684f7df2f5b041f2b020c307c13e437b, user_id=ad3dcaf8bebe42359e2e14de59b267f3, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:27.926542 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1789633f-992a-46d1-86f0-499fe8b02868 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:27.990220 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 588/1153] 10.222.0.98 () {58 vars in 1439 bytes} [Thu Aug 18 12:03:27 2022] PUT /identity/v3/projects/684f7df2f5b041f2b020c307c13e437b/users/ad3dcaf8bebe42359e2e14de59b267f3/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:03:29.914205 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd7ee0-e040-46e2-9c40-ac2941116665 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:29.914757 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd7ee0-e040-46e2-9c40-ac2941116665 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:29.914757 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd7ee0-e040-46e2-9c40-ac2941116665 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:29.916125 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 566/1154] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:03:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:03:29.921371 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45496d-5e40-4c47-b1d5-b1b29473f191 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:29.921784 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45496d-5e40-4c47-b1d5-b1b29473f191 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:29.921920 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45496d-5e40-4c47-b1d5-b1b29473f191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:29.940200 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-4a45496d-5e40-4c47-b1d5-b1b29473f191 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:03:29.941371 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-4a45496d-5e40-4c47-b1d5-b1b29473f191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:03:30.007204 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 589/1155] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:03:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:03:30.275375 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd7ee0-e040-46e2-9c40-ac2941116665 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:30.374435 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd7ee0-e040-46e2-9c40-ac2941116665 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:30.376777 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:30.408508 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32e427b-c3f1-47fd-83c3-2ea574411696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:30.409140 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e427b-c3f1-47fd-83c3-2ea574411696 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:30.409423 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e427b-c3f1-47fd-83c3-2ea574411696 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:30.409698 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e427b-c3f1-47fd-83c3-2ea574411696 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:30.419626 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32e427b-c3f1-47fd-83c3-2ea574411696 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:03:30.424391 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-a32e427b-c3f1-47fd-83c3-2ea574411696 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:03:30.424391 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:03:30.424391 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:03:30.424391 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:03:30.424391 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:03:30.424391 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:03:30.424391 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:03:30.424391 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:03:30.424391 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:03:30.424391 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:03:30.424391 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:03:30.424391 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:03:30.424391 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:03:30.424391 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:03:30.424391 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:03:30.424391 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:03:30.425534 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:03:30.425534 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:03:30.425534 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:03:30.425534 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:03:30.425534 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:03:30.425534 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:03:30.425534 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:03:30.425534 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:03:30.425534 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:03:30.425534 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:03:30.425534 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:03:30.425534 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:03:30.425534 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:03:30.425534 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:03:30.425534 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:03:30.425534 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:03:30.426412 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:03:30.426412 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:03:30.426412 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:03:30.426412 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:03:30.426412 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:03:30.426412 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:03:30.426412 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:03:30.426412 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:03:30.426412 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:03:30.426412 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:03:30.426412 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:03:30.426412 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:03:30.426412 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 567/1156] 10.222.0.98 () {56 vars in 1096 bytes} [Thu Aug 18 12:03:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:03:30.431852 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45496d-5e40-4c47-b1d5-b1b29473f191 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:30.494875 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:30.533642 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45496d-5e40-4c47-b1d5-b1b29473f191 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:30.567518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcda2ee-5a8a-4aa4-b55e-b8e1e7335d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:30.568132 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcda2ee-5a8a-4aa4-b55e-b8e1e7335d4b None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:30.568420 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcda2ee-5a8a-4aa4-b55e-b8e1e7335d4b None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:30.568696 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcda2ee-5a8a-4aa4-b55e-b8e1e7335d4b None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:30.569919 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcda2ee-5a8a-4aa4-b55e-b8e1e7335d4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:03:30.570229 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcda2ee-5a8a-4aa4-b55e-b8e1e7335d4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:30.571616 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcda2ee-5a8a-4aa4-b55e-b8e1e7335d4b None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:30.579525 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 590/1157] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:03:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:30.585887 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32e427b-c3f1-47fd-83c3-2ea574411696 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:30.683735 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32e427b-c3f1-47fd-83c3-2ea574411696 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:30.720738 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643fcdc-9f6c-4f5d-8e40-7c6f383b4d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:30.721371 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643fcdc-9f6c-4f5d-8e40-7c6f383b4d32 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:30.721656 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643fcdc-9f6c-4f5d-8e40-7c6f383b4d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:30.721935 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643fcdc-9f6c-4f5d-8e40-7c6f383b4d32 None admin] PATH_INFO: `/v3/users/ad3dcaf8bebe42359e2e14de59b267f3` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:30.723068 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643fcdc-9f6c-4f5d-8e40-7c6f383b4d32 None admin] RBAC: Authorizing `identity:get_user(user_id=ad3dcaf8bebe42359e2e14de59b267f3)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:30.724673 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643fcdc-9f6c-4f5d-8e40-7c6f383b4d32 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:30.726379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 568/1158] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:03:30 2022] GET /identity/v3/users/ad3dcaf8bebe42359e2e14de59b267f3 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:30.732426 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcda2ee-5a8a-4aa4-b55e-b8e1e7335d4b None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:30.831706 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcda2ee-5a8a-4aa4-b55e-b8e1e7335d4b None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:30.857745 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:30.866283 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc804d-97c7-495e-b28a-1a1d76ffc6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:30.866896 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc804d-97c7-495e-b28a-1a1d76ffc6cd None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:30.875104 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc804d-97c7-495e-b28a-1a1d76ffc6cd None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:30.875104 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc804d-97c7-495e-b28a-1a1d76ffc6cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:30.933149 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bc804d-97c7-495e-b28a-1a1d76ffc6cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:30.934602 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bc804d-97c7-495e-b28a-1a1d76ffc6cd None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:31.031718 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 591/1159] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:03:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 12:03:31.037675 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643fcdc-9f6c-4f5d-8e40-7c6f383b4d32 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:31.071714 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:31.138261 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643fcdc-9f6c-4f5d-8e40-7c6f383b4d32 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:31.172232 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bfccf7-2a98-4133-bae0-d075ab38a0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:31.172856 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bfccf7-2a98-4133-bae0-d075ab38a0de None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:31.173135 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bfccf7-2a98-4133-bae0-d075ab38a0de None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:31.173407 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bfccf7-2a98-4133-bae0-d075ab38a0de None admin] PATH_INFO: `/v3/projects/684f7df2f5b041f2b020c307c13e437b` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:31.174515 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bfccf7-2a98-4133-bae0-d075ab38a0de None admin] RBAC: Authorizing `identity:get_project(project_id=684f7df2f5b041f2b020c307c13e437b)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:31.175974 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bfccf7-2a98-4133-bae0-d075ab38a0de None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:31.177587 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 569/1160] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:03:31 2022] GET /identity/v3/projects/684f7df2f5b041f2b020c307c13e437b => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:31.183743 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc804d-97c7-495e-b28a-1a1d76ffc6cd None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:31.209497 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:31.280776 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc804d-97c7-495e-b28a-1a1d76ffc6cd None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:31.313493 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c18279-fc24-44ce-9ee1-e50b140df793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:31.314134 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c18279-fc24-44ce-9ee1-e50b140df793 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:31.314378 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c18279-fc24-44ce-9ee1-e50b140df793 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:31.314653 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c18279-fc24-44ce-9ee1-e50b140df793 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:31.315662 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c18279-fc24-44ce-9ee1-e50b140df793 None admin] RBAC: Adding query filter params (role.id=c53a0e13b58642f291c856efe1f0b87d, scope.project.id=684f7df2f5b041f2b020c307c13e437b, user.id=ad3dcaf8bebe42359e2e14de59b267f3) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:03:31.315761 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c18279-fc24-44ce-9ee1-e50b140df793 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:31.317118 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c18279-fc24-44ce-9ee1-e50b140df793 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:31.323330 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 592/1161] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:03:31 2022] GET /identity/v3/role_assignments?user.id=ad3dcaf8bebe42359e2e14de59b267f3&scope.project.id=684f7df2f5b041f2b020c307c13e437b&role.id=c53a0e13b58642f291c856efe1f0b87d => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:03:33.218109 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d515a24-8064-40a8-a7a9-ee710e8cc9fe None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:33.218676 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d515a24-8064-40a8-a7a9-ee710e8cc9fe None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:33.218676 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d515a24-8064-40a8-a7a9-ee710e8cc9fe None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:33.220038 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 570/1162] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:03:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:03:33.225256 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfac0e-2c01-4d0c-b614-bbdc833f2244 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:33.226201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfac0e-2c01-4d0c-b614-bbdc833f2244 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:33.226463 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfac0e-2c01-4d0c-b614-bbdc833f2244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:33.244142 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-51cfac0e-2c01-4d0c-b614-bbdc833f2244 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:03:33.245331 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-51cfac0e-2c01-4d0c-b614-bbdc833f2244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:03:33.311372 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 593/1163] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:03:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:03:33.581376 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d515a24-8064-40a8-a7a9-ee710e8cc9fe None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:33.677733 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:33.681985 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d515a24-8064-40a8-a7a9-ee710e8cc9fe None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:33.716178 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e20c3-66da-4918-917c-dd4cde868273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:33.716789 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e20c3-66da-4918-917c-dd4cde868273 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:33.717074 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e20c3-66da-4918-917c-dd4cde868273 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:33.717351 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e20c3-66da-4918-917c-dd4cde868273 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:33.783977 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326e20c3-66da-4918-917c-dd4cde868273 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:33.785448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326e20c3-66da-4918-917c-dd4cde868273 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:33.877522 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:33.881068 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 571/1164] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:03:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 12:03:33.887149 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cfac0e-2c01-4d0c-b614-bbdc833f2244 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:33.908650 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:33.986322 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cfac0e-2c01-4d0c-b614-bbdc833f2244 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:34.018757 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9801154e-5142-44fa-ae1a-77d7f0b225cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:34.019375 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9801154e-5142-44fa-ae1a-77d7f0b225cc None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:34.019683 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9801154e-5142-44fa-ae1a-77d7f0b225cc None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:34.019957 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9801154e-5142-44fa-ae1a-77d7f0b225cc None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:34.026824 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9801154e-5142-44fa-ae1a-77d7f0b225cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:34.028288 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9801154e-5142-44fa-ae1a-77d7f0b225cc None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:34.040063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-9801154e-5142-44fa-ae1a-77d7f0b225cc None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 18 12:03:34.040063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:03:34.040063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 18 12:03:34.040063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 18 12:03:34.040063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:03:34.040063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:03:34.040063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:03:34.040063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:03:34.040063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 18 12:03:34.040063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:03:34.040063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 12:03:34.040063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:03:34.040063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:03:34.040063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:03:34.040063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:03:34.040063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:03:34.040063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:03:34.041379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:03:34.041379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:03:34.041379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:03:34.041379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:03:34.041379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:03:34.041379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:03:34.041379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:03:34.041379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:03:34.041379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:03:34.041379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:03:34.041379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:03:34.041379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:03:34.041379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:03:34.041379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:03:34.041379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:03:34.041379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:03:34.042271 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:03:34.042271 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:03:34.042271 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:03:34.042271 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:03:34.042271 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:03:34.042271 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:03:34.042271 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:03:34.042271 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:03:34.042271 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:03:34.042271 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:03:34.042271 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 18 12:03:34.042271 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:03:34.042271 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 18 12:03:34.042271 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:03:34.042271 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 594/1165] 10.222.0.98 () {56 vars in 1117 bytes} [Thu Aug 18 12:03:33 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:03:34.047355 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e20c3-66da-4918-917c-dd4cde868273 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:34.143333 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e20c3-66da-4918-917c-dd4cde868273 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:34.176232 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ac3ef-e43d-4826-8ec5-d9645d9bfb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:34.176827 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ac3ef-e43d-4826-8ec5-d9645d9bfb27 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:34.177109 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ac3ef-e43d-4826-8ec5-d9645d9bfb27 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:34.177383 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ac3ef-e43d-4826-8ec5-d9645d9bfb27 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:34.178015 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2ac3ef-e43d-4826-8ec5-d9645d9bfb27 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:03:34.178317 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2ac3ef-e43d-4826-8ec5-d9645d9bfb27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:34.179727 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2ac3ef-e43d-4826-8ec5-d9645d9bfb27 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:34.186604 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 572/1166] 10.222.0.98 () {56 vars in 1115 bytes} [Thu Aug 18 12:03:34 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:36.069290 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252837e5-f2e7-4b59-b33f-5195ad141e17 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:36.070444 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252837e5-f2e7-4b59-b33f-5195ad141e17 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:36.070444 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252837e5-f2e7-4b59-b33f-5195ad141e17 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:36.071758 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 595/1167] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:03:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:03:36.076970 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50440bb5-e67f-47ac-a743-19b7b236aa00 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:36.077367 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50440bb5-e67f-47ac-a743-19b7b236aa00 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:36.077532 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50440bb5-e67f-47ac-a743-19b7b236aa00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:36.095156 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-50440bb5-e67f-47ac-a743-19b7b236aa00 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:03:36.096386 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-50440bb5-e67f-47ac-a743-19b7b236aa00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:03:36.153720 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:36.161704 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 573/1168] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:03:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:03:36.430637 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252837e5-f2e7-4b59-b33f-5195ad141e17 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:36.512619 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:36.534046 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252837e5-f2e7-4b59-b33f-5195ad141e17 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:36.568048 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf3a571-4b86-49bd-9700-1fccda43b107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:36.568650 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3a571-4b86-49bd-9700-1fccda43b107 None admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:36.568934 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3a571-4b86-49bd-9700-1fccda43b107 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:36.569208 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3a571-4b86-49bd-9700-1fccda43b107 None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:36.570743 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf3a571-4b86-49bd-9700-1fccda43b107 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:36.572159 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf3a571-4b86-49bd-9700-1fccda43b107 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:36.660103 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 596/1169] 10.222.0.98 () {60 vars in 1138 bytes} [Thu Aug 18 12:03:36 2022] POST /identity/v3/domains => generated 267 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:03:38.574481 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e7be7-3a36-40a1-a315-fc14ee472c2b None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:38.575064 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e7be7-3a36-40a1-a315-fc14ee472c2b None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:38.575064 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e7be7-3a36-40a1-a315-fc14ee472c2b None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:38.576406 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 574/1170] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:03:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:03:38.581619 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d618d-2d04-4c04-92b4-19e76aab63a8 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:38.582052 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d618d-2d04-4c04-92b4-19e76aab63a8 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:38.582184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d618d-2d04-4c04-92b4-19e76aab63a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:38.600065 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-755d618d-2d04-4c04-92b4-19e76aab63a8 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:03:38.601942 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-755d618d-2d04-4c04-92b4-19e76aab63a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:03:38.671012 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 597/1171] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:03:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 12:03:38.949271 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e7be7-3a36-40a1-a315-fc14ee472c2b None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:39.049487 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e7be7-3a36-40a1-a315-fc14ee472c2b None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:39.081846 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c926341f-df98-4307-809d-43e735f974b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:39.082461 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926341f-df98-4307-809d-43e735f974b3 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:39.082746 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926341f-df98-4307-809d-43e735f974b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:39.083025 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926341f-df98-4307-809d-43e735f974b3 None admin] PATH_INFO: `/v3/domains/b64108988a244b99b616e9b59b672b13` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:39.093770 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c926341f-df98-4307-809d-43e735f974b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=b64108988a244b99b616e9b59b672b13)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:39.095209 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c926341f-df98-4307-809d-43e735f974b3 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:39.096826 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 575/1172] 10.222.0.98 () {56 vars in 1183 bytes} [Thu Aug 18 12:03:38 2022] GET /identity/v3/domains/b64108988a244b99b616e9b59b672b13 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:39.103540 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755d618d-2d04-4c04-92b4-19e76aab63a8 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:39.170988 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:39.199659 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755d618d-2d04-4c04-92b4-19e76aab63a8 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:39.234331 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea28a4f-4241-4297-bb2c-32704c588368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:39.234946 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea28a4f-4241-4297-bb2c-32704c588368 None admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:39.235225 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea28a4f-4241-4297-bb2c-32704c588368 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:39.235544 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea28a4f-4241-4297-bb2c-32704c588368 None admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:39.236398 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea28a4f-4241-4297-bb2c-32704c588368 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:39.238535 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea28a4f-4241-4297-bb2c-32704c588368 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:39.312857 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 598/1173] 10.222.0.98 () {60 vars in 1141 bytes} [Thu Aug 18 12:03:39 2022] POST /identity/v3/projects => generated 361 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:03:41.258071 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d249b9c-c6c7-40ac-8a01-2e10ef9e7e92 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:41.258655 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d249b9c-c6c7-40ac-8a01-2e10ef9e7e92 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:41.258655 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d249b9c-c6c7-40ac-8a01-2e10ef9e7e92 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:41.260023 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 576/1174] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:03:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:03:41.267016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878263f6-fe92-4927-9633-45b031b1af4a None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:41.268331 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878263f6-fe92-4927-9633-45b031b1af4a None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:41.268645 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878263f6-fe92-4927-9633-45b031b1af4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:41.287989 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-878263f6-fe92-4927-9633-45b031b1af4a None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:03:41.289225 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-878263f6-fe92-4927-9633-45b031b1af4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:03:41.356431 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 599/1175] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:03:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:03:41.625294 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d249b9c-c6c7-40ac-8a01-2e10ef9e7e92 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:41.684723 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:41.732218 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d249b9c-c6c7-40ac-8a01-2e10ef9e7e92 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:41.765607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf66b5c-6849-4ce1-bf42-66d0c5a712c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:41.766235 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf66b5c-6849-4ce1-bf42-66d0c5a712c2 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:41.766562 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf66b5c-6849-4ce1-bf42-66d0c5a712c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:41.766854 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf66b5c-6849-4ce1-bf42-66d0c5a712c2 None admin] PATH_INFO: `/v3/domains/b64108988a244b99b616e9b59b672b13` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:41.768044 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf66b5c-6849-4ce1-bf42-66d0c5a712c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=b64108988a244b99b616e9b59b672b13)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:41.769484 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf66b5c-6849-4ce1-bf42-66d0c5a712c2 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:41.771043 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 577/1176] 10.222.0.98 () {56 vars in 1183 bytes} [Thu Aug 18 12:03:41 2022] GET /identity/v3/domains/b64108988a244b99b616e9b59b672b13 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:03:41.777407 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878263f6-fe92-4927-9633-45b031b1af4a None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:41.829377 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:41.875199 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878263f6-fe92-4927-9633-45b031b1af4a None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:41.908353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183804d-a120-45af-abe9-d14debc8b5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:41.908909 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183804d-a120-45af-abe9-d14debc8b5a0 None admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:41.909188 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183804d-a120-45af-abe9-d14debc8b5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:41.909462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183804d-a120-45af-abe9-d14debc8b5a0 None admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:41.910163 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a183804d-a120-45af-abe9-d14debc8b5a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:41.911993 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a183804d-a120-45af-abe9-d14debc8b5a0 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:42.037371 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 600/1177] 10.222.0.98 () {60 vars in 1132 bytes} [Thu Aug 18 12:03:41 2022] POST /identity/v3/users => generated 311 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:03:43.981876 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2ffed-5b75-4ff8-90c4-b35cf79fa47e None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:43.982443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2ffed-5b75-4ff8-90c4-b35cf79fa47e None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:43.982443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2ffed-5b75-4ff8-90c4-b35cf79fa47e None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:43.983806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 578/1178] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:03:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:03:43.989747 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51853b-4f24-408e-833d-3a2321d31267 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:43.990203 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51853b-4f24-408e-833d-3a2321d31267 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:43.990612 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51853b-4f24-408e-833d-3a2321d31267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:44.014130 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-4d51853b-4f24-408e-833d-3a2321d31267 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:03:44.015311 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-4d51853b-4f24-408e-833d-3a2321d31267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:03:44.079367 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 601/1179] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:03:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:03:44.376986 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba2ffed-5b75-4ff8-90c4-b35cf79fa47e None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:44.477547 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba2ffed-5b75-4ff8-90c4-b35cf79fa47e None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:44.499482 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:44.511031 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b844caaa-a8c3-4cb3-9824-abbda2d4cf25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:44.522854 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844caaa-a8c3-4cb3-9824-abbda2d4cf25 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:44.522854 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844caaa-a8c3-4cb3-9824-abbda2d4cf25 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:44.522854 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844caaa-a8c3-4cb3-9824-abbda2d4cf25 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:44.522854 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b844caaa-a8c3-4cb3-9824-abbda2d4cf25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:03:44.527348 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-b844caaa-a8c3-4cb3-9824-abbda2d4cf25 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:03:44.527348 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:03:44.527348 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:03:44.527348 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:03:44.527348 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:03:44.527348 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:03:44.527348 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:03:44.527348 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:03:44.527348 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:03:44.527348 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:03:44.527348 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:03:44.527348 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:03:44.527348 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:03:44.527348 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:03:44.527348 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:03:44.527348 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:03:44.528512 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:03:44.528512 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:03:44.528512 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:03:44.528512 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:03:44.528512 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:03:44.528512 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:03:44.528512 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:03:44.528512 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:03:44.528512 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:03:44.528512 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:03:44.528512 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:03:44.528512 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:03:44.528512 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:03:44.528512 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:03:44.528512 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:03:44.528512 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:03:44.529775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:03:44.529775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:03:44.529775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:03:44.529775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:03:44.529775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:03:44.529775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:03:44.529775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:03:44.529775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:03:44.529775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:03:44.529775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:03:44.529775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:03:44.529775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:03:44.529775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 579/1180] 10.222.0.98 () {56 vars in 1096 bytes} [Thu Aug 18 12:03:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:03:44.534943 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51853b-4f24-408e-833d-3a2321d31267 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:44.574779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:44.633951 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51853b-4f24-408e-833d-3a2321d31267 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:44.668910 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13869ae-5c75-4496-8696-7f886f64cfd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:44.669514 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13869ae-5c75-4496-8696-7f886f64cfd4 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:44.669806 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13869ae-5c75-4496-8696-7f886f64cfd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:44.670090 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13869ae-5c75-4496-8696-7f886f64cfd4 None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:44.670759 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13869ae-5c75-4496-8696-7f886f64cfd4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:03:44.671068 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13869ae-5c75-4496-8696-7f886f64cfd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:44.672475 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13869ae-5c75-4496-8696-7f886f64cfd4 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:44.681554 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 602/1181] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:03:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:44.687613 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b844caaa-a8c3-4cb3-9824-abbda2d4cf25 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:44.788537 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b844caaa-a8c3-4cb3-9824-abbda2d4cf25 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:44.823057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d87a58-3cc1-49b4-b844-43830a694d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:44.823721 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d87a58-3cc1-49b4-b844-43830a694d78 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:44.824008 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d87a58-3cc1-49b4-b844-43830a694d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:44.824287 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d87a58-3cc1-49b4-b844-43830a694d78 None admin] PATH_INFO: `/v3/users/9a76c72ea3f64f6790b43584f703ea3b` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:44.838002 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d87a58-3cc1-49b4-b844-43830a694d78 None admin] RBAC: Authorizing `identity:get_user(user_id=9a76c72ea3f64f6790b43584f703ea3b)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:44.839502 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d87a58-3cc1-49b4-b844-43830a694d78 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:44.841108 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 580/1182] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:03:44 2022] GET /identity/v3/users/9a76c72ea3f64f6790b43584f703ea3b => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:44.847085 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13869ae-5c75-4496-8696-7f886f64cfd4 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:44.947513 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:44.950338 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13869ae-5c75-4496-8696-7f886f64cfd4 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:44.982463 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d298a47a-0fb4-4746-b581-2dc1f82d2a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:44.983060 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298a47a-0fb4-4746-b581-2dc1f82d2a4f None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:44.983346 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298a47a-0fb4-4746-b581-2dc1f82d2a4f None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:44.983679 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298a47a-0fb4-4746-b581-2dc1f82d2a4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:45.048282 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d298a47a-0fb4-4746-b581-2dc1f82d2a4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:45.049725 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d298a47a-0fb4-4746-b581-2dc1f82d2a4f None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:45.144434 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:45.148443 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 603/1183] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:03:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 12:03:45.154410 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d87a58-3cc1-49b4-b844-43830a694d78 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:45.196656 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:45.252344 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d87a58-3cc1-49b4-b844-43830a694d78 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:45.285672 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d40e6-2e6e-4cc5-a527-87c4a787bd58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:45.286300 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d40e6-2e6e-4cc5-a527-87c4a787bd58 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:45.286584 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d40e6-2e6e-4cc5-a527-87c4a787bd58 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:45.286861 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d40e6-2e6e-4cc5-a527-87c4a787bd58 None admin] PATH_INFO: `/v3/projects/8f9aac8996204e339c87c710808a90a4` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:45.288005 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d40e6-2e6e-4cc5-a527-87c4a787bd58 None admin] RBAC: Authorizing `identity:get_project(project_id=8f9aac8996204e339c87c710808a90a4)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:45.289483 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d40e6-2e6e-4cc5-a527-87c4a787bd58 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:45.291094 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 581/1184] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:03:45 2022] GET /identity/v3/projects/8f9aac8996204e339c87c710808a90a4 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:45.297255 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d298a47a-0fb4-4746-b581-2dc1f82d2a4f None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:45.393090 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d298a47a-0fb4-4746-b581-2dc1f82d2a4f None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:45.426312 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3025105-e09d-4273-81d9-88503f7eb624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:45.426911 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3025105-e09d-4273-81d9-88503f7eb624 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:45.427194 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3025105-e09d-4273-81d9-88503f7eb624 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:45.427500 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3025105-e09d-4273-81d9-88503f7eb624 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:45.428217 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3025105-e09d-4273-81d9-88503f7eb624 None admin] RBAC: Adding query filter params (role.id=c53a0e13b58642f291c856efe1f0b87d, scope.project.id=8f9aac8996204e339c87c710808a90a4, user.id=9a76c72ea3f64f6790b43584f703ea3b) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:03:45.428529 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3025105-e09d-4273-81d9-88503f7eb624 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:45.429987 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3025105-e09d-4273-81d9-88503f7eb624 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:45.435251 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 604/1185] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:03:45 2022] GET /identity/v3/role_assignments?user.id=9a76c72ea3f64f6790b43584f703ea3b&scope.project.id=8f9aac8996204e339c87c710808a90a4&role.id=c53a0e13b58642f291c856efe1f0b87d => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:47.292154 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d269c-35a6-4cac-ba12-27153061cfdb None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:47.292713 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d269c-35a6-4cac-ba12-27153061cfdb None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:47.292713 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d269c-35a6-4cac-ba12-27153061cfdb None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:47.294045 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 582/1186] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:03:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:03:47.299189 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29ed10-e6ae-41e7-a8d5-f1e22d6e6b78 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:47.299623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29ed10-e6ae-41e7-a8d5-f1e22d6e6b78 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:47.299796 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29ed10-e6ae-41e7-a8d5-f1e22d6e6b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:47.317603 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-df29ed10-e6ae-41e7-a8d5-f1e22d6e6b78 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:03:47.318794 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-df29ed10-e6ae-41e7-a8d5-f1e22d6e6b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:03:47.376075 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:47.384194 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 605/1187] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:03:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:03:47.651297 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d269c-35a6-4cac-ba12-27153061cfdb None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:47.750044 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d269c-35a6-4cac-ba12-27153061cfdb None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:47.759793 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:47.785048 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51941048-e827-4151-bfbc-588408a02688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:47.785668 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51941048-e827-4151-bfbc-588408a02688 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:47.785946 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51941048-e827-4151-bfbc-588408a02688 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:47.786226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51941048-e827-4151-bfbc-588408a02688 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:47.796161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51941048-e827-4151-bfbc-588408a02688 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:03:47.800903 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-51941048-e827-4151-bfbc-588408a02688 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:03:47.800903 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:03:47.800903 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:03:47.800903 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:03:47.800903 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:03:47.800903 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:03:47.800903 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:03:47.800903 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:03:47.800903 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:03:47.800903 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:03:47.800903 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:03:47.800903 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:03:47.800903 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:03:47.800903 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:03:47.800903 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:03:47.800903 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:03:47.801991 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:03:47.801991 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:03:47.801991 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:03:47.801991 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:03:47.801991 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:03:47.801991 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:03:47.801991 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:03:47.801991 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:03:47.801991 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:03:47.801991 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:03:47.801991 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:03:47.801991 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:03:47.801991 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:03:47.801991 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:03:47.801991 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:03:47.801991 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:03:47.803181 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:03:47.803181 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:03:47.803181 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:03:47.803181 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:03:47.803181 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:03:47.803181 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:03:47.803181 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:03:47.803181 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:03:47.803181 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:03:47.803181 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:03:47.803181 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:03:47.803181 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:03:47.803181 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 583/1188] 10.222.0.98 () {56 vars in 1096 bytes} [Thu Aug 18 12:03:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:03:47.808095 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df29ed10-e6ae-41e7-a8d5-f1e22d6e6b78 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:47.903685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df29ed10-e6ae-41e7-a8d5-f1e22d6e6b78 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:47.936687 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f86ef-78e9-4153-90c8-b7cdb5679886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:47.937284 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f86ef-78e9-4153-90c8-b7cdb5679886 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:47.937565 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f86ef-78e9-4153-90c8-b7cdb5679886 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:47.937837 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f86ef-78e9-4153-90c8-b7cdb5679886 None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:47.938527 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f86ef-78e9-4153-90c8-b7cdb5679886 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:03:47.938858 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f86ef-78e9-4153-90c8-b7cdb5679886 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:47.940240 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f86ef-78e9-4153-90c8-b7cdb5679886 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:47.948685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 606/1189] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:03:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:47.954493 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51941048-e827-4151-bfbc-588408a02688 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:48.051219 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51941048-e827-4151-bfbc-588408a02688 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:48.085514 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f21274f-7e9c-46e5-8b84-fe0f8b31dc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:48.086168 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21274f-7e9c-46e5-8b84-fe0f8b31dc61 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:48.086456 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21274f-7e9c-46e5-8b84-fe0f8b31dc61 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:48.086730 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21274f-7e9c-46e5-8b84-fe0f8b31dc61 None admin] PATH_INFO: `/v3/users/9a76c72ea3f64f6790b43584f703ea3b` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:48.087890 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f21274f-7e9c-46e5-8b84-fe0f8b31dc61 None admin] RBAC: Authorizing `identity:get_user(user_id=9a76c72ea3f64f6790b43584f703ea3b)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:48.089315 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f21274f-7e9c-46e5-8b84-fe0f8b31dc61 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:48.090882 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 584/1190] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:03:47 2022] GET /identity/v3/users/9a76c72ea3f64f6790b43584f703ea3b => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:48.096947 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f86ef-78e9-4153-90c8-b7cdb5679886 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:48.152199 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:48.194256 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f86ef-78e9-4153-90c8-b7cdb5679886 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:48.226765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0802435-1a30-4b56-a220-4b04502b6937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:48.227392 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0802435-1a30-4b56-a220-4b04502b6937 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:48.227701 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0802435-1a30-4b56-a220-4b04502b6937 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:48.227973 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0802435-1a30-4b56-a220-4b04502b6937 None admin] PATH_INFO: `/v3/projects/8f9aac8996204e339c87c710808a90a4` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:48.229086 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0802435-1a30-4b56-a220-4b04502b6937 None admin] RBAC: Authorizing `identity:get_project(project_id=8f9aac8996204e339c87c710808a90a4)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:48.230505 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0802435-1a30-4b56-a220-4b04502b6937 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:48.232138 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 607/1191] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:03:48 2022] GET /identity/v3/projects/8f9aac8996204e339c87c710808a90a4 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:48.238140 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f21274f-7e9c-46e5-8b84-fe0f8b31dc61 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:48.273155 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:48.337432 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f21274f-7e9c-46e5-8b84-fe0f8b31dc61 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:48.371570 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e3489e-aded-45cc-abbd-3af7518c3ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:48.372223 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3489e-aded-45cc-abbd-3af7518c3ea1 None admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:48.372507 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3489e-aded-45cc-abbd-3af7518c3ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:48.372784 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3489e-aded-45cc-abbd-3af7518c3ea1 None admin] PATH_INFO: `/v3/projects/8f9aac8996204e339c87c710808a90a4/users/9a76c72ea3f64f6790b43584f703ea3b/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:48.374688 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e3489e-aded-45cc-abbd-3af7518c3ea1 None admin] RBAC: Authorizing `identity:create_grant(project_id=8f9aac8996204e339c87c710808a90a4, user_id=9a76c72ea3f64f6790b43584f703ea3b, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:48.376151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e3489e-aded-45cc-abbd-3af7518c3ea1 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:48.460485 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 585/1192] 10.222.0.98 () {58 vars in 1439 bytes} [Thu Aug 18 12:03:48 2022] PUT /identity/v3/projects/8f9aac8996204e339c87c710808a90a4/users/9a76c72ea3f64f6790b43584f703ea3b/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:03:50.342210 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad820b8-4a29-4c79-84b6-1e03a0d00045 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:50.342779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad820b8-4a29-4c79-84b6-1e03a0d00045 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:50.342779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad820b8-4a29-4c79-84b6-1e03a0d00045 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:50.344173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 608/1193] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:03:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:03:50.349435 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d61be-b0dc-4a2b-9340-958c9d6a49a8 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:50.349881 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d61be-b0dc-4a2b-9340-958c9d6a49a8 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:50.349999 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d61be-b0dc-4a2b-9340-958c9d6a49a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:50.368407 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-b24d61be-b0dc-4a2b-9340-958c9d6a49a8 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:03:50.369613 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-b24d61be-b0dc-4a2b-9340-958c9d6a49a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:03:50.435595 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 586/1194] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:03:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:03:50.702199 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad820b8-4a29-4c79-84b6-1e03a0d00045 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:50.810721 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad820b8-4a29-4c79-84b6-1e03a0d00045 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:50.834175 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:50.845297 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0ea1a-f607-4a7f-afdf-82e0efcda6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:50.847019 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0ea1a-f607-4a7f-afdf-82e0efcda6ec None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:50.847019 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0ea1a-f607-4a7f-afdf-82e0efcda6ec None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:50.847019 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0ea1a-f607-4a7f-afdf-82e0efcda6ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:50.856819 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce0ea1a-f607-4a7f-afdf-82e0efcda6ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:03:50.861432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-2ce0ea1a-f607-4a7f-afdf-82e0efcda6ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:03:50.861432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:03:50.861432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:03:50.861432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:03:50.861432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:03:50.861432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:03:50.861432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:03:50.861432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:03:50.861432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:03:50.861432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:03:50.861432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:03:50.861432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:03:50.861432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:03:50.861432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:03:50.861432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:03:50.861432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:03:50.862560 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:03:50.862560 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:03:50.862560 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:03:50.862560 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:03:50.862560 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:03:50.862560 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:03:50.862560 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:03:50.862560 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:03:50.862560 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:03:50.862560 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:03:50.862560 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:03:50.862560 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:03:50.862560 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:03:50.862560 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:03:50.862560 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:03:50.862560 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:03:50.863545 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:03:50.863545 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:03:50.863545 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:03:50.863545 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:03:50.863545 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:03:50.863545 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:03:50.863545 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:03:50.863545 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:03:50.863545 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:03:50.863545 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:03:50.863545 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:03:50.863545 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:03:50.863545 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 609/1195] 10.222.0.98 () {56 vars in 1096 bytes} [Thu Aug 18 12:03:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:03:50.868658 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d61be-b0dc-4a2b-9340-958c9d6a49a8 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:50.894806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:50.968282 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d61be-b0dc-4a2b-9340-958c9d6a49a8 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:51.001951 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99cce57-1e69-4e46-8e52-302ce094191a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:51.002575 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99cce57-1e69-4e46-8e52-302ce094191a None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:51.002852 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99cce57-1e69-4e46-8e52-302ce094191a None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:51.003121 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99cce57-1e69-4e46-8e52-302ce094191a None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:51.003808 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99cce57-1e69-4e46-8e52-302ce094191a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:03:51.004109 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99cce57-1e69-4e46-8e52-302ce094191a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:51.005468 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99cce57-1e69-4e46-8e52-302ce094191a None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:51.013272 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 587/1196] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:03:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:51.019071 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0ea1a-f607-4a7f-afdf-82e0efcda6ec None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:51.121268 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0ea1a-f607-4a7f-afdf-82e0efcda6ec None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:51.153652 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f4eff-8ca4-4f9e-a914-f16560dcd809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:51.154264 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f4eff-8ca4-4f9e-a914-f16560dcd809 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:51.154544 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f4eff-8ca4-4f9e-a914-f16560dcd809 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:51.154816 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f4eff-8ca4-4f9e-a914-f16560dcd809 None admin] PATH_INFO: `/v3/users/9a76c72ea3f64f6790b43584f703ea3b` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:51.155944 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705f4eff-8ca4-4f9e-a914-f16560dcd809 None admin] RBAC: Authorizing `identity:get_user(user_id=9a76c72ea3f64f6790b43584f703ea3b)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:51.157363 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705f4eff-8ca4-4f9e-a914-f16560dcd809 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:51.159189 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 610/1197] 10.222.0.98 () {56 vars in 1177 bytes} [Thu Aug 18 12:03:51 2022] GET /identity/v3/users/9a76c72ea3f64f6790b43584f703ea3b => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:51.165163 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99cce57-1e69-4e46-8e52-302ce094191a None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:51.250477 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:51.262650 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99cce57-1e69-4e46-8e52-302ce094191a None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:51.294607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0704dfc9-6253-47df-9664-c511470ee064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:51.295206 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704dfc9-6253-47df-9664-c511470ee064 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:51.295512 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704dfc9-6253-47df-9664-c511470ee064 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:51.295787 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704dfc9-6253-47df-9664-c511470ee064 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:51.368818 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0704dfc9-6253-47df-9664-c511470ee064 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:51.370266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0704dfc9-6253-47df-9664-c511470ee064 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:51.453984 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:51.466923 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 588/1198] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:03:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 12:03:51.472876 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f4eff-8ca4-4f9e-a914-f16560dcd809 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:51.525259 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:03:51.572615 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f4eff-8ca4-4f9e-a914-f16560dcd809 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:51.605699 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78392390-939b-4ea9-b826-e57244dc2a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:51.606330 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78392390-939b-4ea9-b826-e57244dc2a30 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:51.606616 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78392390-939b-4ea9-b826-e57244dc2a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:51.606889 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78392390-939b-4ea9-b826-e57244dc2a30 None admin] PATH_INFO: `/v3/projects/8f9aac8996204e339c87c710808a90a4` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:51.608025 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78392390-939b-4ea9-b826-e57244dc2a30 None admin] RBAC: Authorizing `identity:get_project(project_id=8f9aac8996204e339c87c710808a90a4)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:51.609447 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78392390-939b-4ea9-b826-e57244dc2a30 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:51.611176 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 611/1199] 10.222.0.98 () {56 vars in 1186 bytes} [Thu Aug 18 12:03:51 2022] GET /identity/v3/projects/8f9aac8996204e339c87c710808a90a4 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:03:51.617365 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0704dfc9-6253-47df-9664-c511470ee064 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:03:51.717870 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0704dfc9-6253-47df-9664-c511470ee064 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:03:51.752596 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b625a-0bd0-4490-b68d-b420fb2c475b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:03:51.753194 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b625a-0bd0-4490-b68d-b420fb2c475b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:03:51.753477 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b625a-0bd0-4490-b68d-b420fb2c475b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:03:51.753754 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b625a-0bd0-4490-b68d-b420fb2c475b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:03:51.754477 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b625a-0bd0-4490-b68d-b420fb2c475b None admin] RBAC: Adding query filter params (role.id=c53a0e13b58642f291c856efe1f0b87d, scope.project.id=8f9aac8996204e339c87c710808a90a4, user.id=9a76c72ea3f64f6790b43584f703ea3b) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:03:51.754790 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b625a-0bd0-4490-b68d-b420fb2c475b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:03:51.756420 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b625a-0bd0-4490-b68d-b420fb2c475b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:03:51.762539 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 589/1200] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:03:51 2022] GET /identity/v3/role_assignments?user.id=9a76c72ea3f64f6790b43584f703ea3b&scope.project.id=8f9aac8996204e339c87c710808a90a4&role.id=c53a0e13b58642f291c856efe1f0b87d => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:04:01.601466 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=40 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:04:01.760075 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=74 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:04.520692 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5f947-128b-4bb6-b3fd-6752e8283990 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:04.521301 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5f947-128b-4bb6-b3fd-6752e8283990 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:04.521301 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5f947-128b-4bb6-b3fd-6752e8283990 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:04.522671 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 612/1201] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:05:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:05:04.528352 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7444e6-4587-4807-a409-43c4f5c79cca None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:04.530311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7444e6-4587-4807-a409-43c4f5c79cca None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:04.530311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7444e6-4587-4807-a409-43c4f5c79cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:04.547216 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-ac7444e6-4587-4807-a409-43c4f5c79cca None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:05:04.548454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-ac7444e6-4587-4807-a409-43c4f5c79cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:05:04.615660 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 590/1202] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:05:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:05:04.892234 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5f947-128b-4bb6-b3fd-6752e8283990 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:04.993343 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5f947-128b-4bb6-b3fd-6752e8283990 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:05.027552 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9795f2-113d-47f0-b81a-55a6d364a8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:05.028181 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9795f2-113d-47f0-b81a-55a6d364a8b3 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:05.028465 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9795f2-113d-47f0-b81a-55a6d364a8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:05.028742 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9795f2-113d-47f0-b81a-55a6d364a8b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:05.039479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9795f2-113d-47f0-b81a-55a6d364a8b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:05.041691 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9795f2-113d-47f0-b81a-55a6d364a8b3 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:05.055307 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-bc9795f2-113d-47f0-b81a-55a6d364a8b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:05:05.055307 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:05:05.055307 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:05:05.055307 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:05:05.055307 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:05:05.055307 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:05:05.055307 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:05:05.055307 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:05:05.055307 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:05:05.055307 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:05:05.055307 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:05:05.055307 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:05:05.055307 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:05:05.055307 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:05:05.055307 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:05:05.055307 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:05:05.056636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:05:05.056636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:05:05.056636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:05:05.056636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:05:05.056636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:05:05.056636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:05:05.056636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:05:05.056636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:05:05.056636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:05:05.056636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:05:05.056636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:05:05.056636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:05:05.056636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:05:05.056636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:05:05.056636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:05:05.056636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:05:05.056636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:05:05.057851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:05:05.057851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:05:05.057851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:05:05.057851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 613/1203] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:05:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:05:05.063111 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7444e6-4587-4807-a409-43c4f5c79cca None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:05.162097 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7444e6-4587-4807-a409-43c4f5c79cca None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:05.192916 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:05.198777 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054c0b2e-dbae-480a-b80a-40c695cc035d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:05.199389 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c0b2e-dbae-480a-b80a-40c695cc035d None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:05.199706 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c0b2e-dbae-480a-b80a-40c695cc035d None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:05.199984 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c0b2e-dbae-480a-b80a-40c695cc035d None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:05.200622 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054c0b2e-dbae-480a-b80a-40c695cc035d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:05.200929 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054c0b2e-dbae-480a-b80a-40c695cc035d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:05.202291 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054c0b2e-dbae-480a-b80a-40c695cc035d None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:05.213359 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 591/1204] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:05:05.220031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9795f2-113d-47f0-b81a-55a6d364a8b3 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:05.272976 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:05.320525 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9795f2-113d-47f0-b81a-55a6d364a8b3 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:05.352819 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac6c5b8-a2db-4455-8833-243de61f4794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:05.353429 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6c5b8-a2db-4455-8833-243de61f4794 None admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:05.353755 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6c5b8-a2db-4455-8833-243de61f4794 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:05.354004 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6c5b8-a2db-4455-8833-243de61f4794 None admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:05.354631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac6c5b8-a2db-4455-8833-243de61f4794 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:05.356099 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac6c5b8-a2db-4455-8833-243de61f4794 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:05.449948 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 614/1205] 10.222.0.98 () {60 vars in 1132 bytes} [Thu Aug 18 12:05:05 2022] POST /identity/v3/users => generated 251 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:05:07.409718 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4f13e-3846-49ef-b996-b45973c174fd None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:07.410299 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4f13e-3846-49ef-b996-b45973c174fd None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:07.410299 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4f13e-3846-49ef-b996-b45973c174fd None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:07.411654 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 592/1206] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:05:07.417523 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147c96b-c3ee-4644-95a7-2b6f72edaab1 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:07.420708 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147c96b-c3ee-4644-95a7-2b6f72edaab1 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:07.420708 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147c96b-c3ee-4644-95a7-2b6f72edaab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:07.440298 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-8147c96b-c3ee-4644-95a7-2b6f72edaab1 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:05:07.442290 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-8147c96b-c3ee-4644-95a7-2b6f72edaab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:05:07.513772 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 615/1207] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:05:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:05:07.782789 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4f13e-3846-49ef-b996-b45973c174fd None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:07.884702 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4f13e-3846-49ef-b996-b45973c174fd None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:07.917133 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e48849-b599-4a2e-b7ad-f7d7fc788138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:07.917755 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e48849-b599-4a2e-b7ad-f7d7fc788138 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:07.918037 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e48849-b599-4a2e-b7ad-f7d7fc788138 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:07.918310 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e48849-b599-4a2e-b7ad-f7d7fc788138 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:07.930112 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e48849-b599-4a2e-b7ad-f7d7fc788138 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:05:07.935028 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-65e48849-b599-4a2e-b7ad-f7d7fc788138 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:05:07.935028 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:05:07.935028 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:05:07.935028 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:05:07.935028 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:05:07.935028 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:05:07.935028 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:05:07.935028 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:05:07.935028 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:05:07.935028 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:05:07.935028 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:05:07.935028 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:05:07.935028 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:05:07.935028 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:05:07.935028 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:05:07.935028 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:05:07.936804 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:05:07.936804 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:05:07.936804 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:05:07.936804 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:05:07.936804 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:05:07.936804 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:05:07.936804 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:05:07.936804 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:05:07.936804 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:05:07.936804 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:05:07.936804 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:05:07.936804 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:05:07.936804 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:05:07.936804 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:05:07.936804 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:05:07.936804 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:05:07.937786 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:05:07.937786 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:05:07.937786 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:05:07.937786 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:05:07.937786 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:05:07.937786 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:05:07.937786 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:05:07.937786 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:05:07.937786 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:05:07.937786 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:05:07.937786 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:05:07.937786 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:05:07.937786 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 593/1208] 10.222.0.98 () {56 vars in 1102 bytes} [Thu Aug 18 12:05:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:05:07.943022 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8147c96b-c3ee-4644-95a7-2b6f72edaab1 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:07.982382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:08.042025 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8147c96b-c3ee-4644-95a7-2b6f72edaab1 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:08.077409 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb195d-c2af-4602-be85-1b13f0896222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:08.078061 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb195d-c2af-4602-be85-1b13f0896222 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:08.078332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb195d-c2af-4602-be85-1b13f0896222 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:08.078609 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb195d-c2af-4602-be85-1b13f0896222 None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:08.079284 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eb195d-c2af-4602-be85-1b13f0896222 None admin] RBAC: Adding query filter params (name=service) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:08.079621 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eb195d-c2af-4602-be85-1b13f0896222 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:08.080993 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eb195d-c2af-4602-be85-1b13f0896222 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:08.088821 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 616/1209] 10.222.0.98 () {56 vars in 1103 bytes} [Thu Aug 18 12:05:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:05:08.094730 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e48849-b599-4a2e-b7ad-f7d7fc788138 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:08.145738 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:08.196193 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e48849-b599-4a2e-b7ad-f7d7fc788138 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:08.229507 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a635c-95f2-4fd3-a2b9-488501b5d3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:08.230129 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a635c-95f2-4fd3-a2b9-488501b5d3a1 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:08.230415 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a635c-95f2-4fd3-a2b9-488501b5d3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:08.230693 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a635c-95f2-4fd3-a2b9-488501b5d3a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:08.244792 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21a635c-95f2-4fd3-a2b9-488501b5d3a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:08.244792 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21a635c-95f2-4fd3-a2b9-488501b5d3a1 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:08.257342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-a21a635c-95f2-4fd3-a2b9-488501b5d3a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:05:08.257342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:05:08.257342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:05:08.257342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:05:08.257342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:05:08.257342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:05:08.257342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:05:08.257342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:05:08.257342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:05:08.257342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:05:08.257342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:05:08.257342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:05:08.257342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:05:08.257342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:05:08.257342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:05:08.257342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:05:08.263248 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:05:08.263248 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:05:08.263248 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:05:08.263248 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:05:08.263248 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:05:08.263248 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:05:08.263248 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:05:08.263248 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:05:08.263248 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:05:08.263248 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:05:08.263248 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:05:08.263248 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:05:08.263248 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:05:08.263248 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:05:08.263248 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:05:08.263248 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:05:08.263248 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:05:08.264444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:05:08.264444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:05:08.264444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:05:08.264444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 594/1210] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:05:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:05:08.264746 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb195d-c2af-4602-be85-1b13f0896222 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:08.363430 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:08.366930 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb195d-c2af-4602-be85-1b13f0896222 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:08.399569 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05408f42-6ba4-4349-ada3-f80ad10e2db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:08.400168 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05408f42-6ba4-4349-ada3-f80ad10e2db0 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:08.400460 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05408f42-6ba4-4349-ada3-f80ad10e2db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:08.400717 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05408f42-6ba4-4349-ada3-f80ad10e2db0 None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:08.401345 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05408f42-6ba4-4349-ada3-f80ad10e2db0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:08.401683 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05408f42-6ba4-4349-ada3-f80ad10e2db0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:08.403037 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05408f42-6ba4-4349-ada3-f80ad10e2db0 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:08.415094 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 617/1211] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:05:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:05:08.421042 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a635c-95f2-4fd3-a2b9-488501b5d3a1 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:08.526347 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a635c-95f2-4fd3-a2b9-488501b5d3a1 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:08.527832 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:08.560789 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10933e1a-86d7-4d35-9d23-f22630e16c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:08.561413 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10933e1a-86d7-4d35-9d23-f22630e16c04 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:08.561694 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10933e1a-86d7-4d35-9d23-f22630e16c04 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:08.561967 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10933e1a-86d7-4d35-9d23-f22630e16c04 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:08.569897 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10933e1a-86d7-4d35-9d23-f22630e16c04 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:08.571452 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10933e1a-86d7-4d35-9d23-f22630e16c04 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:08.583285 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-10933e1a-86d7-4d35-9d23-f22630e16c04 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 18 12:05:08.583285 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:05:08.583285 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:05:08.583285 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:05:08.583285 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:05:08.583285 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:05:08.583285 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:05:08.583285 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:05:08.583285 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:05:08.583285 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:05:08.583285 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:05:08.583285 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:05:08.583285 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 12:05:08.583285 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 12:05:08.583285 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 12:05:08.583285 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 12:05:08.584529 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:05:08.584529 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:05:08.584529 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:05:08.584529 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:05:08.584529 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:05:08.584529 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:05:08.584529 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:05:08.584529 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:05:08.584529 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:05:08.584529 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:05:08.584529 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:05:08.584529 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:05:08.584529 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:05:08.584529 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:05:08.584529 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:05:08.584529 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:05:08.584529 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:05:08.584529 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:05:08.585755 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:05:08.585755 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:05:08.585755 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 12:05:08.585755 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 12:05:08.585755 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 12:05:08.585755 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 12:05:08.585755 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 12:05:08.585755 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 12:05:08.585755 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 18 12:05:08.585755 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:05:08.585755 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 595/1212] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:05:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:05:08.590399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05408f42-6ba4-4349-ada3-f80ad10e2db0 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:08.699269 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05408f42-6ba4-4349-ada3-f80ad10e2db0 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:08.735402 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc18338-73c4-496c-a8cd-4e4669162a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:08.736053 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc18338-73c4-496c-a8cd-4e4669162a51 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:08.736327 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc18338-73c4-496c-a8cd-4e4669162a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:08.736605 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc18338-73c4-496c-a8cd-4e4669162a51 None admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:08.737330 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc18338-73c4-496c-a8cd-4e4669162a51 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:08.737642 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc18338-73c4-496c-a8cd-4e4669162a51 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:08.739246 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc18338-73c4-496c-a8cd-4e4669162a51 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:08.756294 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 618/1213] 10.222.0.98 () {56 vars in 1143 bytes} [Thu Aug 18 12:05:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:05:08.762252 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10933e1a-86d7-4d35-9d23-f22630e16c04 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:08.862918 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10933e1a-86d7-4d35-9d23-f22630e16c04 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:08.896363 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d8d4fe-989a-4056-b5b8-cdede5b8acf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:08.896971 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8d4fe-989a-4056-b5b8-cdede5b8acf5 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:08.897248 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8d4fe-989a-4056-b5b8-cdede5b8acf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:08.897519 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8d4fe-989a-4056-b5b8-cdede5b8acf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:08.928459 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:08.977391 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d8d4fe-989a-4056-b5b8-cdede5b8acf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:08.978789 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d8d4fe-989a-4056-b5b8-cdede5b8acf5 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:09.080016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 596/1214] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:05:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 12:05:09.085889 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc18338-73c4-496c-a8cd-4e4669162a51 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:09.104929 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:09.184320 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc18338-73c4-496c-a8cd-4e4669162a51 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:09.220288 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14278d-3667-43f1-9ecb-1e6268d4a058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:09.220942 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14278d-3667-43f1-9ecb-1e6268d4a058 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:09.221213 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14278d-3667-43f1-9ecb-1e6268d4a058 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:09.221487 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14278d-3667-43f1-9ecb-1e6268d4a058 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:09.232054 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd14278d-3667-43f1-9ecb-1e6268d4a058 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:09.233582 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd14278d-3667-43f1-9ecb-1e6268d4a058 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:09.248786 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-bd14278d-3667-43f1-9ecb-1e6268d4a058 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:05:09.248786 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:05:09.248786 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:05:09.248786 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:05:09.248786 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:05:09.248786 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:05:09.248786 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:05:09.248786 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:05:09.248786 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:05:09.248786 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:05:09.248786 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:05:09.248786 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:05:09.248786 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:05:09.248786 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:05:09.248786 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:05:09.248786 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:05:09.250443 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:05:09.250443 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:05:09.250443 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:05:09.250443 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:05:09.250443 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:05:09.250443 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:05:09.250443 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:05:09.250443 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:05:09.250443 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:05:09.250443 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:05:09.250443 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:05:09.250443 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:05:09.250443 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:05:09.250443 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:05:09.250443 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:05:09.250443 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:05:09.250443 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:05:09.251351 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:05:09.251351 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:05:09.251351 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:05:09.251351 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 619/1215] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:05:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:05:09.256310 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d8d4fe-989a-4056-b5b8-cdede5b8acf5 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:09.327354 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:09.365704 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d8d4fe-989a-4056-b5b8-cdede5b8acf5 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:09.399747 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb4f754-2b18-4014-8df5-03028bcda72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:09.400349 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4f754-2b18-4014-8df5-03028bcda72a None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:09.400633 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4f754-2b18-4014-8df5-03028bcda72a None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:09.400905 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4f754-2b18-4014-8df5-03028bcda72a None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:09.401538 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb4f754-2b18-4014-8df5-03028bcda72a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:09.401844 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb4f754-2b18-4014-8df5-03028bcda72a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:09.403194 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb4f754-2b18-4014-8df5-03028bcda72a None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:09.414068 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 597/1216] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:05:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:05:09.420062 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14278d-3667-43f1-9ecb-1e6268d4a058 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:09.496164 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:09.527232 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14278d-3667-43f1-9ecb-1e6268d4a058 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:09.561433 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68deb68c-53f4-4570-8fd9-5b3e47ab5fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:09.562067 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68deb68c-53f4-4570-8fd9-5b3e47ab5fa2 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:09.562377 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68deb68c-53f4-4570-8fd9-5b3e47ab5fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:09.562657 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68deb68c-53f4-4570-8fd9-5b3e47ab5fa2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:09.569398 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68deb68c-53f4-4570-8fd9-5b3e47ab5fa2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:09.570985 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68deb68c-53f4-4570-8fd9-5b3e47ab5fa2 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:09.582728 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-68deb68c-53f4-4570-8fd9-5b3e47ab5fa2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:05:09.582728 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:05:09.582728 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:05:09.582728 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:05:09.582728 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:05:09.582728 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:05:09.582728 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:05:09.582728 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:05:09.582728 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:05:09.582728 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:05:09.582728 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:05:09.582728 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:05:09.582728 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 12:05:09.582728 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 12:05:09.582728 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 12:05:09.583913 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 12:05:09.583913 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:05:09.583913 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:05:09.583913 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:05:09.583913 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:05:09.583913 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:05:09.583913 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:05:09.583913 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:05:09.583913 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:05:09.583913 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:05:09.583913 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:05:09.583913 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:05:09.583913 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:05:09.583913 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:05:09.583913 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:05:09.583913 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:05:09.583913 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:05:09.584836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 12:05:09.584836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 12:05:09.584836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:05:09.584836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:05:09.584836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:05:09.584836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:05:09.584836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:05:09.584836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:05:09.584836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 620/1217] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:05:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:05:09.589957 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb4f754-2b18-4014-8df5-03028bcda72a None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:09.690619 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb4f754-2b18-4014-8df5-03028bcda72a None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:09.714094 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:09.725823 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85b481-95a9-4b16-ac52-7b7b342eda32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:09.726573 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85b481-95a9-4b16-ac52-7b7b342eda32 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:09.726750 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85b481-95a9-4b16-ac52-7b7b342eda32 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:09.727026 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85b481-95a9-4b16-ac52-7b7b342eda32 None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:09.727755 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a85b481-95a9-4b16-ac52-7b7b342eda32 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:09.728060 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a85b481-95a9-4b16-ac52-7b7b342eda32 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:09.730189 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a85b481-95a9-4b16-ac52-7b7b342eda32 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:09.742163 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 598/1218] 10.222.0.98 () {56 vars in 1148 bytes} [Thu Aug 18 12:05:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:05:09.748487 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68deb68c-53f4-4570-8fd9-5b3e47ab5fa2 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:09.849152 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68deb68c-53f4-4570-8fd9-5b3e47ab5fa2 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:09.877169 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:09.882926 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03332723-1d9e-4119-a92c-550eb600a087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:09.883595 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03332723-1d9e-4119-a92c-550eb600a087 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:09.883881 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03332723-1d9e-4119-a92c-550eb600a087 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:09.884157 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03332723-1d9e-4119-a92c-550eb600a087 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:09.884881 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03332723-1d9e-4119-a92c-550eb600a087 None admin] RBAC: Adding query filter params (role.id=563e3ed5c6164aae966b64cc302b114a, scope.project.id=6f676a5169464d83af4267db25025728, user.id=193637acadc14ec4a48ab79566ba3dab) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:09.885196 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03332723-1d9e-4119-a92c-550eb600a087 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:09.886626 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03332723-1d9e-4119-a92c-550eb600a087 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:09.892603 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 621/1219] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:05:09 2022] GET /identity/v3/role_assignments?user.id=193637acadc14ec4a48ab79566ba3dab&scope.project.id=6f676a5169464d83af4267db25025728&role.id=563e3ed5c6164aae966b64cc302b114a => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:05:11.785654 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeab361-31a9-4435-98dd-654b26476f1c None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:11.786212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeab361-31a9-4435-98dd-654b26476f1c None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:11.786212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeab361-31a9-4435-98dd-654b26476f1c None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:11.787621 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 599/1220] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:05:11.793166 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514f8c7-d018-4b2b-b9c9-92ff3a29ccbd None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:11.809816 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514f8c7-d018-4b2b-b9c9-92ff3a29ccbd None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:11.809816 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514f8c7-d018-4b2b-b9c9-92ff3a29ccbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:11.811654 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-b514f8c7-d018-4b2b-b9c9-92ff3a29ccbd None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:05:11.812928 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-b514f8c7-d018-4b2b-b9c9-92ff3a29ccbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:05:11.880377 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 622/1221] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:05:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:05:12.159375 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeab361-31a9-4435-98dd-654b26476f1c None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:12.261446 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeab361-31a9-4435-98dd-654b26476f1c None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:12.296037 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f931d72e-991b-4568-aaba-8aaf0b519b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:12.296670 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931d72e-991b-4568-aaba-8aaf0b519b6b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:12.297437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931d72e-991b-4568-aaba-8aaf0b519b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:12.297706 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931d72e-991b-4568-aaba-8aaf0b519b6b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:12.307810 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f931d72e-991b-4568-aaba-8aaf0b519b6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:05:12.313164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-f931d72e-991b-4568-aaba-8aaf0b519b6b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:05:12.313164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:05:12.313164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:05:12.313164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:05:12.313164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:05:12.313164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:05:12.313164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:05:12.313164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:05:12.313164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:05:12.313164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:05:12.313164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:05:12.313164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:05:12.313164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:05:12.313164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:05:12.313164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:05:12.313164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:05:12.316448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:05:12.316448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:05:12.316448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:05:12.316448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:05:12.316448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:05:12.316448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:05:12.316448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:05:12.316448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:05:12.316448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:05:12.316448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:05:12.316448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:05:12.316448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:05:12.316448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:05:12.316448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:05:12.316448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:05:12.316448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:05:12.317495 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:05:12.317495 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:05:12.317495 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:05:12.317495 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:05:12.317495 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:05:12.317495 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:05:12.317495 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:05:12.317495 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:05:12.317495 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:05:12.317495 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:05:12.317495 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:05:12.317495 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:05:12.317495 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 600/1222] 10.222.0.98 () {56 vars in 1102 bytes} [Thu Aug 18 12:05:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:05:12.320473 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514f8c7-d018-4b2b-b9c9-92ff3a29ccbd None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:12.420601 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514f8c7-d018-4b2b-b9c9-92ff3a29ccbd None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:12.445479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:12.455718 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157d450-24d2-4530-811d-ad9f64ef147c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:12.456327 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157d450-24d2-4530-811d-ad9f64ef147c None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:12.456662 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157d450-24d2-4530-811d-ad9f64ef147c None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:12.456944 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157d450-24d2-4530-811d-ad9f64ef147c None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:12.457598 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e157d450-24d2-4530-811d-ad9f64ef147c None admin] RBAC: Adding query filter params (name=service) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:12.457905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e157d450-24d2-4530-811d-ad9f64ef147c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:12.459269 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e157d450-24d2-4530-811d-ad9f64ef147c None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:12.467543 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 623/1223] 10.222.0.98 () {56 vars in 1103 bytes} [Thu Aug 18 12:05:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:05:12.473236 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f931d72e-991b-4568-aaba-8aaf0b519b6b None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:12.518242 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:12.576450 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f931d72e-991b-4568-aaba-8aaf0b519b6b None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:12.609916 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51646e18-c872-4b5a-9379-452a0c6bb7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:12.610540 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51646e18-c872-4b5a-9379-452a0c6bb7a9 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:12.610838 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51646e18-c872-4b5a-9379-452a0c6bb7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:12.611104 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51646e18-c872-4b5a-9379-452a0c6bb7a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:12.620950 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51646e18-c872-4b5a-9379-452a0c6bb7a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:12.625554 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51646e18-c872-4b5a-9379-452a0c6bb7a9 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:12.637190 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-51646e18-c872-4b5a-9379-452a0c6bb7a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:05:12.637190 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:05:12.637190 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:05:12.637190 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:05:12.637190 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:05:12.637190 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:05:12.637190 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:05:12.637190 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:05:12.637190 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:05:12.637190 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:05:12.637190 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:05:12.637190 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:05:12.637190 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:05:12.637190 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:05:12.637190 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:05:12.637190 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:05:12.638680 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:05:12.638680 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:05:12.638680 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:05:12.638680 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:05:12.638680 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:05:12.638680 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:05:12.638680 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:05:12.638680 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:05:12.638680 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:05:12.638680 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:05:12.638680 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:05:12.638680 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:05:12.638680 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:05:12.638680 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:05:12.638680 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:05:12.638680 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:05:12.638680 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:05:12.640501 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:05:12.640501 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:05:12.640501 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:05:12.640501 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 601/1224] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:05:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:05:12.644268 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157d450-24d2-4530-811d-ad9f64ef147c None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:12.746981 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157d450-24d2-4530-811d-ad9f64ef147c None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:12.779431 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a554b98-1a3a-4b52-bb35-e27ed3c4616f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:12.780042 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a554b98-1a3a-4b52-bb35-e27ed3c4616f None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:12.780330 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a554b98-1a3a-4b52-bb35-e27ed3c4616f None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:12.780780 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a554b98-1a3a-4b52-bb35-e27ed3c4616f None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:12.781945 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a554b98-1a3a-4b52-bb35-e27ed3c4616f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:12.782255 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a554b98-1a3a-4b52-bb35-e27ed3c4616f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:12.783659 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a554b98-1a3a-4b52-bb35-e27ed3c4616f None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:12.796253 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 624/1225] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:05:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:05:12.802691 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51646e18-c872-4b5a-9379-452a0c6bb7a9 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:12.896052 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:12.902651 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51646e18-c872-4b5a-9379-452a0c6bb7a9 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:12.935970 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e760f495-ca44-4cc8-af98-f7d8a0a71891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:12.936595 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760f495-ca44-4cc8-af98-f7d8a0a71891 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:12.936882 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760f495-ca44-4cc8-af98-f7d8a0a71891 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:12.937162 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760f495-ca44-4cc8-af98-f7d8a0a71891 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:12.944661 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e760f495-ca44-4cc8-af98-f7d8a0a71891 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:12.946043 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e760f495-ca44-4cc8-af98-f7d8a0a71891 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:12.957530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-e760f495-ca44-4cc8-af98-f7d8a0a71891 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 18 12:05:12.957530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:05:12.957530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:05:12.957530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:05:12.957530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:05:12.957530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:05:12.957530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:05:12.957530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:05:12.957530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:05:12.957530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:05:12.957530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:05:12.957530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:05:12.957530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 12:05:12.957530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 12:05:12.957530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 12:05:12.957530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 12:05:12.960806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:05:12.960806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:05:12.960806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:05:12.960806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:05:12.960806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:05:12.960806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:05:12.960806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:05:12.960806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:05:12.960806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:05:12.960806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:05:12.960806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:05:12.960806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:05:12.960806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:05:12.960806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:05:12.960806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:05:12.960806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:05:12.960806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:05:12.960806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:05:12.961774 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:05:12.961774 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:05:12.961774 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 12:05:12.961774 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 12:05:12.961774 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 12:05:12.961774 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 12:05:12.961774 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 12:05:12.961774 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 12:05:12.961774 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 18 12:05:12.961774 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:05:12.961774 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 602/1226] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:05:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:05:12.964690 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a554b98-1a3a-4b52-bb35-e27ed3c4616f None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:13.012016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:13.064033 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a554b98-1a3a-4b52-bb35-e27ed3c4616f None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:13.097851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7860e03c-add8-40be-8a51-88592acab8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:13.098446 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860e03c-add8-40be-8a51-88592acab8f7 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:13.099839 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860e03c-add8-40be-8a51-88592acab8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:13.099839 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860e03c-add8-40be-8a51-88592acab8f7 None admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:13.099839 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7860e03c-add8-40be-8a51-88592acab8f7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:13.100147 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7860e03c-add8-40be-8a51-88592acab8f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:13.101531 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7860e03c-add8-40be-8a51-88592acab8f7 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:13.118046 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 625/1227] 10.222.0.98 () {56 vars in 1143 bytes} [Thu Aug 18 12:05:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:05:13.123968 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e760f495-ca44-4cc8-af98-f7d8a0a71891 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:13.224740 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e760f495-ca44-4cc8-af98-f7d8a0a71891 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:13.262639 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e1c43a-b116-4c3f-a9f6-53432bb3591e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:13.263361 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1c43a-b116-4c3f-a9f6-53432bb3591e None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:13.263669 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1c43a-b116-4c3f-a9f6-53432bb3591e None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:13.263944 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1c43a-b116-4c3f-a9f6-53432bb3591e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:13.274847 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e1c43a-b116-4c3f-a9f6-53432bb3591e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:13.276323 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e1c43a-b116-4c3f-a9f6-53432bb3591e None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:13.292485 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-f5e1c43a-b116-4c3f-a9f6-53432bb3591e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:05:13.292485 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:05:13.292485 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:05:13.292485 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:05:13.292485 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:05:13.292485 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:05:13.292485 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:05:13.292485 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:05:13.292485 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:05:13.292485 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:05:13.292485 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:05:13.292485 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:05:13.292485 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:05:13.292485 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:05:13.292485 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:05:13.292485 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:05:13.293682 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:05:13.293682 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:05:13.293682 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:05:13.293682 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:05:13.293682 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:05:13.293682 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:05:13.293682 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:05:13.293682 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:05:13.293682 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:05:13.293682 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:05:13.293682 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:05:13.293682 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:05:13.293682 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:05:13.293682 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:05:13.293682 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:05:13.293682 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:05:13.293682 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:05:13.294998 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:05:13.294998 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:05:13.294998 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:05:13.294998 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 603/1228] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:05:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:05:13.299585 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7860e03c-add8-40be-8a51-88592acab8f7 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:13.401531 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7860e03c-add8-40be-8a51-88592acab8f7 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:13.403948 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:13.435942 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93691b2d-b316-43d9-8352-aea4a249ee21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:13.436559 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93691b2d-b316-43d9-8352-aea4a249ee21 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:13.436841 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93691b2d-b316-43d9-8352-aea4a249ee21 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:13.437115 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93691b2d-b316-43d9-8352-aea4a249ee21 None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:13.438431 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93691b2d-b316-43d9-8352-aea4a249ee21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:13.438738 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93691b2d-b316-43d9-8352-aea4a249ee21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:13.440126 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93691b2d-b316-43d9-8352-aea4a249ee21 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:13.451274 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 626/1229] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:05:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:05:13.457680 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e1c43a-b116-4c3f-a9f6-53432bb3591e None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:13.468072 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:13.561219 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e1c43a-b116-4c3f-a9f6-53432bb3591e None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:13.595325 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a4448-e602-4805-ad4a-adfb334cf4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:13.596017 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a4448-e602-4805-ad4a-adfb334cf4d6 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:13.596300 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a4448-e602-4805-ad4a-adfb334cf4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:13.596575 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a4448-e602-4805-ad4a-adfb334cf4d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:13.603186 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a4448-e602-4805-ad4a-adfb334cf4d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:13.604730 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a4448-e602-4805-ad4a-adfb334cf4d6 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:13.616056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-5e3a4448-e602-4805-ad4a-adfb334cf4d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:05:13.616056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:05:13.616056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:05:13.616056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:05:13.616056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:05:13.616056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:05:13.616056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:05:13.616056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:05:13.616056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:05:13.616056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:05:13.616056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:05:13.616056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:05:13.616056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 12:05:13.616056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 12:05:13.616056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 12:05:13.617090 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 12:05:13.617090 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:05:13.617090 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:05:13.617090 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:05:13.617090 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:05:13.617090 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:05:13.617090 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:05:13.617090 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:05:13.617090 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:05:13.617090 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:05:13.617090 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:05:13.617090 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:05:13.617090 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:05:13.617090 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:05:13.617090 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:05:13.617090 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:05:13.617090 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:05:13.618116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 12:05:13.618116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 12:05:13.618116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:05:13.618116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:05:13.618116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:05:13.618116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:05:13.618116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:05:13.618116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:05:13.618116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 604/1230] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:05:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:05:13.623138 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93691b2d-b316-43d9-8352-aea4a249ee21 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:13.731548 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93691b2d-b316-43d9-8352-aea4a249ee21 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:13.766305 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4aa0d3-f2ae-4d39-8049-686bddcfd3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:13.766913 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4aa0d3-f2ae-4d39-8049-686bddcfd3e6 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:13.767199 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4aa0d3-f2ae-4d39-8049-686bddcfd3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:13.767502 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4aa0d3-f2ae-4d39-8049-686bddcfd3e6 None admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:13.768190 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4aa0d3-f2ae-4d39-8049-686bddcfd3e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:13.768535 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4aa0d3-f2ae-4d39-8049-686bddcfd3e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:13.770067 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4aa0d3-f2ae-4d39-8049-686bddcfd3e6 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:13.782141 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 627/1231] 10.222.0.98 () {56 vars in 1148 bytes} [Thu Aug 18 12:05:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:05:13.788324 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a4448-e602-4805-ad4a-adfb334cf4d6 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:13.855077 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:13.891042 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a4448-e602-4805-ad4a-adfb334cf4d6 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:13.923941 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec3d7a-67d7-41cc-80f1-c3c9c318f53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:13.924593 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec3d7a-67d7-41cc-80f1-c3c9c318f53b None admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:13.924875 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec3d7a-67d7-41cc-80f1-c3c9c318f53b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:13.925227 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec3d7a-67d7-41cc-80f1-c3c9c318f53b None admin] PATH_INFO: `/v3/projects/6f676a5169464d83af4267db25025728/users/193637acadc14ec4a48ab79566ba3dab/roles/563e3ed5c6164aae966b64cc302b114a` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:13.940601 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceec3d7a-67d7-41cc-80f1-c3c9c318f53b None admin] RBAC: Authorizing `identity:create_grant(project_id=6f676a5169464d83af4267db25025728, user_id=193637acadc14ec4a48ab79566ba3dab, role_id=563e3ed5c6164aae966b64cc302b114a)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:13.942038 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceec3d7a-67d7-41cc-80f1-c3c9c318f53b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:14.046264 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 605/1232] 10.222.0.98 () {58 vars in 1439 bytes} [Thu Aug 18 12:05:13 2022] PUT /identity/v3/projects/6f676a5169464d83af4267db25025728/users/193637acadc14ec4a48ab79566ba3dab/roles/563e3ed5c6164aae966b64cc302b114a => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:05:15.972435 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27ad14-3f49-448f-a708-74cfc15edc39 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:15.972989 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27ad14-3f49-448f-a708-74cfc15edc39 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:15.973168 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27ad14-3f49-448f-a708-74cfc15edc39 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:15.974613 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 628/1233] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:05:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:05:15.981161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79f484-bbe4-4e12-92ef-e69715014a42 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:15.981684 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79f484-bbe4-4e12-92ef-e69715014a42 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:15.981824 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79f484-bbe4-4e12-92ef-e69715014a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:16.000333 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-4b79f484-bbe4-4e12-92ef-e69715014a42 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:05:16.001537 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-4b79f484-bbe4-4e12-92ef-e69715014a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:05:16.068391 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 606/1234] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:05:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:05:16.336035 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27ad14-3f49-448f-a708-74cfc15edc39 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:16.361754 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:16.437199 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27ad14-3f49-448f-a708-74cfc15edc39 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:16.472460 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad81ee2-7da7-409a-a776-8580dfc6ca3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:16.473073 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad81ee2-7da7-409a-a776-8580dfc6ca3c None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:16.473352 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad81ee2-7da7-409a-a776-8580dfc6ca3c None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:16.473630 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad81ee2-7da7-409a-a776-8580dfc6ca3c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:16.483839 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad81ee2-7da7-409a-a776-8580dfc6ca3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:05:16.488419 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-1ad81ee2-7da7-409a-a776-8580dfc6ca3c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:05:16.488419 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:05:16.488419 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:05:16.488419 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:05:16.488419 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:05:16.488419 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:05:16.488419 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:05:16.488419 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:05:16.488419 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:05:16.488419 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:05:16.488419 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:05:16.488419 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:05:16.488419 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:05:16.488419 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:05:16.488419 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:05:16.488419 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:05:16.489631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:05:16.489631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:05:16.489631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:05:16.489631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:05:16.489631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:05:16.489631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:05:16.489631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:05:16.489631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:05:16.489631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:05:16.489631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:05:16.489631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:05:16.489631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:05:16.489631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:05:16.489631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:05:16.489631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:05:16.489631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:05:16.490638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:05:16.490638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:05:16.490638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:05:16.490638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:05:16.490638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:05:16.490638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:05:16.490638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:05:16.490638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:05:16.490638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:05:16.490638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:05:16.490638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 18 12:05:16.490638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:05:16.490638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 629/1235] 10.222.0.98 () {56 vars in 1102 bytes} [Thu Aug 18 12:05:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:05:16.495599 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b79f484-bbe4-4e12-92ef-e69715014a42 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:16.543276 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:16.598194 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b79f484-bbe4-4e12-92ef-e69715014a42 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:16.631906 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c2aae-208b-4977-9ae9-aeefb6b09b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:16.632494 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c2aae-208b-4977-9ae9-aeefb6b09b98 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:16.632832 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c2aae-208b-4977-9ae9-aeefb6b09b98 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:16.633111 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c2aae-208b-4977-9ae9-aeefb6b09b98 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:16.633782 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c2aae-208b-4977-9ae9-aeefb6b09b98 None admin] RBAC: Adding query filter params (name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:16.634093 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c2aae-208b-4977-9ae9-aeefb6b09b98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:16.635476 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c2aae-208b-4977-9ae9-aeefb6b09b98 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:16.644302 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 607/1236] 10.222.0.98 () {56 vars in 1103 bytes} [Thu Aug 18 12:05:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:05:16.649804 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad81ee2-7da7-409a-a776-8580dfc6ca3c None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:16.740426 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:16.760170 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad81ee2-7da7-409a-a776-8580dfc6ca3c None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:16.794625 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1732b3a1-24e3-4a6d-9bf6-250378b25bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:16.795253 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732b3a1-24e3-4a6d-9bf6-250378b25bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:16.795578 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732b3a1-24e3-4a6d-9bf6-250378b25bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:16.795897 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732b3a1-24e3-4a6d-9bf6-250378b25bc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:16.807067 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1732b3a1-24e3-4a6d-9bf6-250378b25bc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:16.808699 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1732b3a1-24e3-4a6d-9bf6-250378b25bc7 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:16.823462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-1732b3a1-24e3-4a6d-9bf6-250378b25bc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:05:16.823462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:05:16.823462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:05:16.823462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:05:16.823462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:05:16.823462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:05:16.823462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:05:16.823462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:05:16.823462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:05:16.823462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:05:16.823462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:05:16.823462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:05:16.823462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:05:16.823462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:05:16.823462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:05:16.823462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:05:16.824586 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:05:16.824586 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:05:16.824586 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:05:16.824586 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:05:16.824586 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:05:16.824586 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:05:16.824586 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:05:16.824586 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:05:16.824586 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:05:16.824586 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:05:16.824586 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:05:16.824586 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:05:16.824586 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:05:16.824586 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:05:16.824586 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:05:16.824586 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:05:16.824586 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:05:16.825568 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:05:16.825568 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:05:16.825568 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:05:16.825568 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 630/1237] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:05:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:05:16.830725 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c2aae-208b-4977-9ae9-aeefb6b09b98 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:16.931060 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c2aae-208b-4977-9ae9-aeefb6b09b98 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:16.939744 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:16.964326 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86381852-65a5-4075-84af-1166e45694d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:16.964937 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86381852-65a5-4075-84af-1166e45694d4 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:16.965222 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86381852-65a5-4075-84af-1166e45694d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:16.965499 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86381852-65a5-4075-84af-1166e45694d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:16.966139 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86381852-65a5-4075-84af-1166e45694d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:16.966443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86381852-65a5-4075-84af-1166e45694d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:16.967842 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86381852-65a5-4075-84af-1166e45694d4 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:16.981066 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 608/1238] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:05:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:05:16.986970 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1732b3a1-24e3-4a6d-9bf6-250378b25bc7 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:17.090179 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1732b3a1-24e3-4a6d-9bf6-250378b25bc7 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:17.128858 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81da4b8-3b50-41e9-ab01-6e100bc219b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:17.129486 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81da4b8-3b50-41e9-ab01-6e100bc219b3 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:17.129768 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81da4b8-3b50-41e9-ab01-6e100bc219b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:17.130045 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81da4b8-3b50-41e9-ab01-6e100bc219b3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:17.132712 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:17.137793 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81da4b8-3b50-41e9-ab01-6e100bc219b3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:17.139226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81da4b8-3b50-41e9-ab01-6e100bc219b3 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:17.150415 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-f81da4b8-3b50-41e9-ab01-6e100bc219b3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 18 12:05:17.150415 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:05:17.150415 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:05:17.150415 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:05:17.150415 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:05:17.150415 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:05:17.150415 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:05:17.150415 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:05:17.150415 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:05:17.150415 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:05:17.150415 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:05:17.150415 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:05:17.150415 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 12:05:17.150415 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 12:05:17.150415 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 12:05:17.150415 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 12:05:17.151605 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:05:17.151605 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:05:17.151605 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:05:17.151605 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:05:17.151605 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:05:17.151605 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:05:17.151605 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:05:17.151605 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:05:17.151605 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:05:17.151605 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:05:17.151605 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:05:17.151605 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:05:17.151605 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:05:17.151605 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:05:17.151605 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:05:17.151605 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:05:17.151605 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:05:17.151605 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:05:17.152793 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:05:17.152793 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:05:17.152793 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 12:05:17.152793 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 12:05:17.152793 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 12:05:17.152793 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 12:05:17.152793 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 12:05:17.152793 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 12:05:17.152793 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 18 12:05:17.152793 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:05:17.152793 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 631/1239] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:05:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:05:17.158017 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86381852-65a5-4075-84af-1166e45694d4 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:17.261714 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86381852-65a5-4075-84af-1166e45694d4 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:17.295179 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3f77dd-d7a3-41b2-a3d5-0b70f4097ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:17.295897 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f77dd-d7a3-41b2-a3d5-0b70f4097ee5 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:17.296243 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f77dd-d7a3-41b2-a3d5-0b70f4097ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:17.296568 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f77dd-d7a3-41b2-a3d5-0b70f4097ee5 None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:17.297602 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3f77dd-d7a3-41b2-a3d5-0b70f4097ee5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:17.297953 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3f77dd-d7a3-41b2-a3d5-0b70f4097ee5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:17.299404 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3f77dd-d7a3-41b2-a3d5-0b70f4097ee5 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:17.316580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 609/1240] 10.222.0.98 () {56 vars in 1143 bytes} [Thu Aug 18 12:05:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:05:17.322459 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81da4b8-3b50-41e9-ab01-6e100bc219b3 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:17.432182 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81da4b8-3b50-41e9-ab01-6e100bc219b3 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:17.466356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f69238a-8006-4d5e-adba-a4a03def170f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:17.466958 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69238a-8006-4d5e-adba-a4a03def170f None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:17.467240 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69238a-8006-4d5e-adba-a4a03def170f None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:17.467544 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69238a-8006-4d5e-adba-a4a03def170f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:17.528328 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:17.534892 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f69238a-8006-4d5e-adba-a4a03def170f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:17.536439 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f69238a-8006-4d5e-adba-a4a03def170f None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:17.636028 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 632/1241] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:05:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 12:05:17.642690 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3f77dd-d7a3-41b2-a3d5-0b70f4097ee5 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:17.670940 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:17.742635 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3f77dd-d7a3-41b2-a3d5-0b70f4097ee5 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:17.775374 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f42871-5e07-4564-b8d9-4260da86d917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:17.776087 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f42871-5e07-4564-b8d9-4260da86d917 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:17.776372 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f42871-5e07-4564-b8d9-4260da86d917 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:17.776649 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f42871-5e07-4564-b8d9-4260da86d917 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:17.789238 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f42871-5e07-4564-b8d9-4260da86d917 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:17.789238 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f42871-5e07-4564-b8d9-4260da86d917 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:17.802831 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-41f42871-5e07-4564-b8d9-4260da86d917 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:05:17.802831 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:05:17.802831 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:05:17.802831 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:05:17.802831 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:05:17.802831 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:05:17.802831 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:05:17.802831 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:05:17.802831 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:05:17.802831 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:05:17.802831 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:05:17.802831 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:05:17.802831 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:05:17.802831 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:05:17.802831 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:05:17.802831 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:05:17.803900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:05:17.803900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:05:17.803900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:05:17.803900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:05:17.803900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:05:17.803900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:05:17.803900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:05:17.803900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:05:17.803900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:05:17.803900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:05:17.803900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:05:17.803900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:05:17.803900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:05:17.803900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:05:17.803900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:05:17.803900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:05:17.803900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:05:17.804867 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:05:17.804867 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:05:17.804867 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:05:17.804867 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 610/1242] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:05:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:05:17.810049 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f69238a-8006-4d5e-adba-a4a03def170f None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:17.904827 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:17.911219 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f69238a-8006-4d5e-adba-a4a03def170f None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:17.945329 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485aa4d4-3154-4794-aae3-79e04667fac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:17.945933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485aa4d4-3154-4794-aae3-79e04667fac6 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:17.946220 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485aa4d4-3154-4794-aae3-79e04667fac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:17.946553 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485aa4d4-3154-4794-aae3-79e04667fac6 None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:17.947195 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485aa4d4-3154-4794-aae3-79e04667fac6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:17.947534 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485aa4d4-3154-4794-aae3-79e04667fac6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:17.948898 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485aa4d4-3154-4794-aae3-79e04667fac6 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:17.960944 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 633/1243] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:05:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:05:17.966885 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f42871-5e07-4564-b8d9-4260da86d917 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:18.048890 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:18.071587 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f42871-5e07-4564-b8d9-4260da86d917 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:18.105467 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca53425-c49f-48e4-89d5-10f0645074ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:18.106122 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca53425-c49f-48e4-89d5-10f0645074ba None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:18.106381 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca53425-c49f-48e4-89d5-10f0645074ba None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:18.106657 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca53425-c49f-48e4-89d5-10f0645074ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:18.113225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca53425-c49f-48e4-89d5-10f0645074ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:18.114665 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca53425-c49f-48e4-89d5-10f0645074ba None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:18.126115 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-2ca53425-c49f-48e4-89d5-10f0645074ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:05:18.126115 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:05:18.126115 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:05:18.126115 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:05:18.126115 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:05:18.126115 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:05:18.126115 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:05:18.126115 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:05:18.126115 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:05:18.126115 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:05:18.126115 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:05:18.126115 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:05:18.126115 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 12:05:18.126115 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 12:05:18.126115 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 12:05:18.127220 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 12:05:18.127220 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:05:18.127220 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:05:18.127220 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:05:18.127220 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:05:18.127220 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:05:18.127220 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:05:18.127220 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:05:18.127220 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:05:18.127220 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:05:18.127220 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:05:18.127220 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:05:18.127220 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:05:18.127220 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:05:18.127220 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:05:18.127220 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:05:18.127220 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:05:18.128693 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 12:05:18.128693 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 12:05:18.128693 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:05:18.128693 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:05:18.128693 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:05:18.128693 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:05:18.128693 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:05:18.128693 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:05:18.128693 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 611/1244] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:05:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:05:18.133423 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485aa4d4-3154-4794-aae3-79e04667fac6 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:18.235587 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485aa4d4-3154-4794-aae3-79e04667fac6 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:18.270064 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02838059-eec6-4add-b359-27f7adab7523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:18.270671 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02838059-eec6-4add-b359-27f7adab7523 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:18.270957 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02838059-eec6-4add-b359-27f7adab7523 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:18.271239 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02838059-eec6-4add-b359-27f7adab7523 None admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:18.271959 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02838059-eec6-4add-b359-27f7adab7523 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:18.272272 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02838059-eec6-4add-b359-27f7adab7523 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:18.273694 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02838059-eec6-4add-b359-27f7adab7523 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:18.285604 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 634/1245] 10.222.0.98 () {56 vars in 1148 bytes} [Thu Aug 18 12:05:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:05:18.292059 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca53425-c49f-48e4-89d5-10f0645074ba None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:18.392782 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca53425-c49f-48e4-89d5-10f0645074ba None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:18.427306 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29138903-7c4c-45be-a62f-47e241bbbaca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:18.427939 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29138903-7c4c-45be-a62f-47e241bbbaca None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:18.428221 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29138903-7c4c-45be-a62f-47e241bbbaca None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:18.428496 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29138903-7c4c-45be-a62f-47e241bbbaca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:18.429307 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29138903-7c4c-45be-a62f-47e241bbbaca None admin] RBAC: Adding query filter params (role.id=563e3ed5c6164aae966b64cc302b114a, scope.project.id=6f676a5169464d83af4267db25025728, user.id=193637acadc14ec4a48ab79566ba3dab) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:18.429617 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29138903-7c4c-45be-a62f-47e241bbbaca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:18.431040 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29138903-7c4c-45be-a62f-47e241bbbaca None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:18.436870 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 612/1246] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:05:18 2022] GET /identity/v3/role_assignments?user.id=193637acadc14ec4a48ab79566ba3dab&scope.project.id=6f676a5169464d83af4267db25025728&role.id=563e3ed5c6164aae966b64cc302b114a => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:05:20.412785 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f5bbce-5c0f-4744-a912-1f1c6e34558c None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:20.413317 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f5bbce-5c0f-4744-a912-1f1c6e34558c None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:20.413407 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f5bbce-5c0f-4744-a912-1f1c6e34558c None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:20.414695 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 635/1247] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:05:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:05:20.420479 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2d2b8-eed6-4aeb-81a4-1fd590998e4f None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:20.420898 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2d2b8-eed6-4aeb-81a4-1fd590998e4f None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:20.421040 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2d2b8-eed6-4aeb-81a4-1fd590998e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:20.427323 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:20.440748 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-bea2d2b8-eed6-4aeb-81a4-1fd590998e4f None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:05:20.442086 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-bea2d2b8-eed6-4aeb-81a4-1fd590998e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:05:20.510271 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 613/1248] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:05:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:05:20.779224 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f5bbce-5c0f-4744-a912-1f1c6e34558c None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:20.799164 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:20.882082 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f5bbce-5c0f-4744-a912-1f1c6e34558c None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:20.915547 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a2134-6221-4697-9cff-41a6f7dc1663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:20.916171 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a2134-6221-4697-9cff-41a6f7dc1663 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:20.916453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a2134-6221-4697-9cff-41a6f7dc1663 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:20.916727 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a2134-6221-4697-9cff-41a6f7dc1663 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:20.927127 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a2134-6221-4697-9cff-41a6f7dc1663 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:05:20.932272 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-552a2134-6221-4697-9cff-41a6f7dc1663 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:05:20.932272 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:05:20.932272 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:05:20.932272 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:05:20.932272 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:05:20.932272 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:05:20.932272 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:05:20.932272 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:05:20.932272 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:05:20.932272 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:05:20.932272 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:05:20.932272 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:05:20.932272 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:05:20.932272 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:05:20.932272 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:05:20.932272 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:05:20.933439 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:05:20.933439 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:05:20.933439 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:05:20.933439 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:05:20.933439 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:05:20.933439 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:05:20.933439 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:05:20.933439 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:05:20.933439 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:05:20.933439 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:05:20.933439 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:05:20.933439 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:05:20.933439 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:05:20.933439 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:05:20.933439 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:05:20.933439 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:05:20.934574 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:05:20.934574 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:05:20.934574 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:05:20.934574 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:05:20.934574 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:05:20.934574 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:05:20.934574 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:05:20.934574 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:05:20.934574 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:05:20.934574 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:05:20.934574 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:05:20.934574 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:05:20.935678 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 636/1249] 10.222.0.98 () {56 vars in 1096 bytes} [Thu Aug 18 12:05:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:05:20.939607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea2d2b8-eed6-4aeb-81a4-1fd590998e4f None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:21.037835 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea2d2b8-eed6-4aeb-81a4-1fd590998e4f None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:21.066796 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:21.071523 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f833a75-e894-4c4a-9da2-db6981f571dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:21.072121 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f833a75-e894-4c4a-9da2-db6981f571dd None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:21.072399 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f833a75-e894-4c4a-9da2-db6981f571dd None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:21.072666 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f833a75-e894-4c4a-9da2-db6981f571dd None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:21.073342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f833a75-e894-4c4a-9da2-db6981f571dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:21.073623 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f833a75-e894-4c4a-9da2-db6981f571dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:21.074966 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f833a75-e894-4c4a-9da2-db6981f571dd None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:21.082699 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 614/1250] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:05:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:05:21.088777 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a2134-6221-4697-9cff-41a6f7dc1663 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:21.167466 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:21.190350 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a2134-6221-4697-9cff-41a6f7dc1663 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:21.224518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5231d5-a650-4251-896b-3c5a481dc6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:21.225150 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5231d5-a650-4251-896b-3c5a481dc6bf None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:21.226578 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5231d5-a650-4251-896b-3c5a481dc6bf None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:21.226578 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5231d5-a650-4251-896b-3c5a481dc6bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:21.237240 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5231d5-a650-4251-896b-3c5a481dc6bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:21.238686 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5231d5-a650-4251-896b-3c5a481dc6bf None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:21.252379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-0a5231d5-a650-4251-896b-3c5a481dc6bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:05:21.252379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:05:21.252379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:05:21.252379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:05:21.252379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:05:21.252379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:05:21.252379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:05:21.252379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:05:21.252379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:05:21.252379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:05:21.252379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:05:21.252379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:05:21.252379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:05:21.252379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:05:21.252379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:05:21.252379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:05:21.253465 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:05:21.253465 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:05:21.253465 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:05:21.253465 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:05:21.253465 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:05:21.253465 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:05:21.253465 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:05:21.253465 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:05:21.253465 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:05:21.253465 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:05:21.253465 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:05:21.253465 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:05:21.253465 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:05:21.253465 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:05:21.253465 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:05:21.253465 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:05:21.253465 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:05:21.254389 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:05:21.254389 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:05:21.254389 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:05:21.254389 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 637/1251] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:05:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:05:21.259625 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f833a75-e894-4c4a-9da2-db6981f571dd None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:21.360710 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f833a75-e894-4c4a-9da2-db6981f571dd None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:21.393657 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee474797-6dc1-4404-ac9e-361deec01c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:21.394250 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee474797-6dc1-4404-ac9e-361deec01c25 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:21.394530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee474797-6dc1-4404-ac9e-361deec01c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:21.394802 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee474797-6dc1-4404-ac9e-361deec01c25 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:21.395452 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee474797-6dc1-4404-ac9e-361deec01c25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:21.395759 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee474797-6dc1-4404-ac9e-361deec01c25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:21.397111 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee474797-6dc1-4404-ac9e-361deec01c25 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:21.409494 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 615/1252] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:05:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:05:21.415823 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5231d5-a650-4251-896b-3c5a481dc6bf None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:21.525988 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5231d5-a650-4251-896b-3c5a481dc6bf None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:21.553465 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:21.561271 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeabff1-050c-4032-a758-5488feb0c49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:21.561904 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeabff1-050c-4032-a758-5488feb0c49f None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:21.562190 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeabff1-050c-4032-a758-5488feb0c49f None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:21.562469 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeabff1-050c-4032-a758-5488feb0c49f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:21.570099 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeabff1-050c-4032-a758-5488feb0c49f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:21.571582 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeabff1-050c-4032-a758-5488feb0c49f None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:21.583173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-bfeabff1-050c-4032-a758-5488feb0c49f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 18 12:05:21.583173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:05:21.583173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:05:21.583173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:05:21.583173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:05:21.583173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:05:21.583173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:05:21.583173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:05:21.583173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:05:21.583173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:05:21.583173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:05:21.583173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:05:21.583173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 12:05:21.583173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 12:05:21.583173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 12:05:21.583173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 12:05:21.584361 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:05:21.584361 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:05:21.584361 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:05:21.584361 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:05:21.584361 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:05:21.584361 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:05:21.584361 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:05:21.584361 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:05:21.584361 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:05:21.584361 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:05:21.584361 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:05:21.584361 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:05:21.584361 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:05:21.584361 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:05:21.584361 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:05:21.584361 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:05:21.584361 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:05:21.584361 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:05:21.585715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:05:21.585715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:05:21.585715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 12:05:21.585715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 12:05:21.585715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 12:05:21.585715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 12:05:21.585715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 12:05:21.585715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 12:05:21.585715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 18 12:05:21.585715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:05:21.585715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 638/1253] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:05:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:05:21.590194 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee474797-6dc1-4404-ac9e-361deec01c25 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:21.644486 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:21.690167 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee474797-6dc1-4404-ac9e-361deec01c25 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:21.724820 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95eb92-7d79-472c-9475-d9a74bf0441a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:21.725420 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95eb92-7d79-472c-9475-d9a74bf0441a None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:21.725699 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95eb92-7d79-472c-9475-d9a74bf0441a None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:21.725970 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95eb92-7d79-472c-9475-d9a74bf0441a None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:21.726696 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95eb92-7d79-472c-9475-d9a74bf0441a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:21.726999 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95eb92-7d79-472c-9475-d9a74bf0441a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:21.728469 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95eb92-7d79-472c-9475-d9a74bf0441a None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:21.745473 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 616/1254] 10.222.0.98 () {56 vars in 1143 bytes} [Thu Aug 18 12:05:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:05:21.751891 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeabff1-050c-4032-a758-5488feb0c49f None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:21.851869 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeabff1-050c-4032-a758-5488feb0c49f None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:21.884767 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0eb8e6-944b-4bbd-ae26-9dcc9f9fc07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:21.885374 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0eb8e6-944b-4bbd-ae26-9dcc9f9fc07f None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:21.885654 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0eb8e6-944b-4bbd-ae26-9dcc9f9fc07f None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:21.885927 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0eb8e6-944b-4bbd-ae26-9dcc9f9fc07f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:21.940231 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:21.952054 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0eb8e6-944b-4bbd-ae26-9dcc9f9fc07f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:21.953521 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0eb8e6-944b-4bbd-ae26-9dcc9f9fc07f None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:22.049724 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 639/1255] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:05:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 12:05:22.055809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95eb92-7d79-472c-9475-d9a74bf0441a None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:22.157420 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95eb92-7d79-472c-9475-d9a74bf0441a None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:22.170660 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:22.192134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e1f57-c450-4786-abf5-097df75781a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:22.192685 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e1f57-c450-4786-abf5-097df75781a1 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:22.193013 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e1f57-c450-4786-abf5-097df75781a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:22.193296 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e1f57-c450-4786-abf5-097df75781a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:22.203509 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e1f57-c450-4786-abf5-097df75781a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:22.205560 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e1f57-c450-4786-abf5-097df75781a1 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:22.221118 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-447e1f57-c450-4786-abf5-097df75781a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:05:22.221118 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:05:22.221118 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:05:22.221118 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:05:22.221118 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:05:22.221118 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:05:22.221118 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:05:22.221118 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:05:22.221118 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:05:22.221118 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:05:22.221118 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:05:22.221118 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:05:22.221118 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:05:22.221118 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:05:22.221118 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:05:22.221118 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:05:22.222266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:05:22.222266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:05:22.222266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:05:22.222266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:05:22.222266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:05:22.222266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:05:22.222266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:05:22.222266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:05:22.222266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:05:22.222266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:05:22.222266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:05:22.222266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:05:22.222266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:05:22.222266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:05:22.222266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:05:22.222266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:05:22.222266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:05:22.223226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:05:22.223226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:05:22.223226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:05:22.223787 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 617/1256] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:05:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:05:22.228098 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0eb8e6-944b-4bbd-ae26-9dcc9f9fc07f None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:22.316295 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:22.327967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0eb8e6-944b-4bbd-ae26-9dcc9f9fc07f None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:22.363078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603027ae-fed5-43c7-a488-50d603e59baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:22.363708 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603027ae-fed5-43c7-a488-50d603e59baf None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:22.363993 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603027ae-fed5-43c7-a488-50d603e59baf None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:22.364270 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603027ae-fed5-43c7-a488-50d603e59baf None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:22.364913 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603027ae-fed5-43c7-a488-50d603e59baf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:22.365216 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603027ae-fed5-43c7-a488-50d603e59baf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:22.366574 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603027ae-fed5-43c7-a488-50d603e59baf None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:22.377689 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 640/1257] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:05:22.383458 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e1f57-c450-4786-abf5-097df75781a1 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:22.482273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e1f57-c450-4786-abf5-097df75781a1 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:22.515095 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c720c0c-f9ee-46e2-b465-766ae9075877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:22.515742 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c720c0c-f9ee-46e2-b465-766ae9075877 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:22.516026 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c720c0c-f9ee-46e2-b465-766ae9075877 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:22.516883 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c720c0c-f9ee-46e2-b465-766ae9075877 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:22.523801 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c720c0c-f9ee-46e2-b465-766ae9075877 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:22.525255 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c720c0c-f9ee-46e2-b465-766ae9075877 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:22.536747 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-0c720c0c-f9ee-46e2-b465-766ae9075877 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:05:22.536747 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:05:22.536747 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:05:22.536747 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:05:22.536747 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:05:22.536747 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:05:22.536747 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:05:22.536747 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:05:22.536747 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:05:22.536747 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:05:22.536747 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:05:22.536747 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:05:22.536747 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 12:05:22.536747 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 12:05:22.536747 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 12:05:22.537922 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 12:05:22.537922 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:05:22.537922 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:05:22.537922 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:05:22.537922 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:05:22.537922 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:05:22.537922 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:05:22.537922 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:05:22.537922 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:05:22.537922 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:05:22.537922 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:05:22.537922 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:05:22.537922 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:05:22.537922 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:05:22.537922 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:05:22.537922 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:05:22.537922 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:05:22.539000 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 12:05:22.539000 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 12:05:22.539000 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:05:22.539000 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:05:22.539000 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:05:22.539000 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:05:22.539000 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:05:22.539000 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:05:22.539000 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 618/1258] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:05:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:05:22.544216 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603027ae-fed5-43c7-a488-50d603e59baf None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:22.646388 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603027ae-fed5-43c7-a488-50d603e59baf None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:22.680570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06160bac-efa5-418c-ae93-395f64cc9daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:22.681184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06160bac-efa5-418c-ae93-395f64cc9daa None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:22.681469 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06160bac-efa5-418c-ae93-395f64cc9daa None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:22.681748 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06160bac-efa5-418c-ae93-395f64cc9daa None admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:22.682442 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06160bac-efa5-418c-ae93-395f64cc9daa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:22.682755 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06160bac-efa5-418c-ae93-395f64cc9daa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:22.684219 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06160bac-efa5-418c-ae93-395f64cc9daa None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:22.805173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 641/1259] 10.222.0.98 () {56 vars in 1148 bytes} [Thu Aug 18 12:05:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:05:22.805380 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c720c0c-f9ee-46e2-b465-766ae9075877 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:22.805380 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:22.805380 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c720c0c-f9ee-46e2-b465-766ae9075877 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:22.837310 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a3994a-4126-4231-8701-f36fcf9c8f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:22.837952 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3994a-4126-4231-8701-f36fcf9c8f77 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:22.838234 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3994a-4126-4231-8701-f36fcf9c8f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:22.838510 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3994a-4126-4231-8701-f36fcf9c8f77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:22.839234 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a3994a-4126-4231-8701-f36fcf9c8f77 None admin] RBAC: Adding query filter params (role.id=c53a0e13b58642f291c856efe1f0b87d, scope.project.id=6f676a5169464d83af4267db25025728, user.id=193637acadc14ec4a48ab79566ba3dab) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:22.839578 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a3994a-4126-4231-8701-f36fcf9c8f77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:22.841002 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a3994a-4126-4231-8701-f36fcf9c8f77 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:22.847183 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 619/1260] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:05:22 2022] GET /identity/v3/role_assignments?user.id=193637acadc14ec4a48ab79566ba3dab&scope.project.id=6f676a5169464d83af4267db25025728&role.id=c53a0e13b58642f291c856efe1f0b87d => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:05:24.738913 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6f71d-34ef-46fd-9b1e-79f6d10b90ea None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:24.739663 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6f71d-34ef-46fd-9b1e-79f6d10b90ea None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:24.739663 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6f71d-34ef-46fd-9b1e-79f6d10b90ea None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:24.741019 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 642/1261] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:05:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:05:24.746288 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fdac88-9e8e-4eea-a28d-45085b1956af None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:24.746869 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fdac88-9e8e-4eea-a28d-45085b1956af None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:24.746869 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fdac88-9e8e-4eea-a28d-45085b1956af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:24.766322 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-e5fdac88-9e8e-4eea-a28d-45085b1956af None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:05:24.772151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-e5fdac88-9e8e-4eea-a28d-45085b1956af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:05:24.834211 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 620/1262] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:05:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 18 12:05:25.097903 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6f71d-34ef-46fd-9b1e-79f6d10b90ea None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:25.109261 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:25.199052 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6f71d-34ef-46fd-9b1e-79f6d10b90ea None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:25.232487 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419de711-213d-4455-b3f0-9e74b2b676be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:25.233115 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419de711-213d-4455-b3f0-9e74b2b676be None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:25.233401 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419de711-213d-4455-b3f0-9e74b2b676be None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:25.233683 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419de711-213d-4455-b3f0-9e74b2b676be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:25.244108 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419de711-213d-4455-b3f0-9e74b2b676be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:05:25.248727 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-419de711-213d-4455-b3f0-9e74b2b676be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:05:25.248727 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:05:25.248727 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:05:25.248727 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:05:25.248727 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:05:25.248727 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:05:25.248727 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:05:25.248727 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:05:25.248727 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:05:25.248727 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:05:25.248727 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:05:25.248727 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:05:25.248727 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:05:25.248727 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:05:25.248727 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:05:25.248727 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:05:25.250066 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:05:25.250066 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:05:25.250066 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:05:25.250066 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:05:25.250066 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:05:25.250066 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:05:25.250066 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:05:25.250066 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:05:25.250066 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:05:25.250066 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:05:25.250066 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:05:25.250066 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:05:25.250066 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:05:25.250066 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:05:25.250066 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:05:25.250066 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:05:25.251899 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:05:25.251899 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:05:25.251899 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:05:25.251899 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:05:25.251899 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:05:25.251899 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:05:25.251899 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:05:25.251899 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:05:25.251899 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:05:25.251899 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:05:25.251899 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:05:25.251899 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:05:25.251899 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 643/1263] 10.222.0.98 () {56 vars in 1096 bytes} [Thu Aug 18 12:05:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:05:25.256500 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fdac88-9e8e-4eea-a28d-45085b1956af None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:25.274509 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:25.354168 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fdac88-9e8e-4eea-a28d-45085b1956af None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:25.389624 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75738429-0f82-46f7-b2be-ecddabcf2458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:25.390231 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75738429-0f82-46f7-b2be-ecddabcf2458 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:25.390512 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75738429-0f82-46f7-b2be-ecddabcf2458 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:25.390786 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75738429-0f82-46f7-b2be-ecddabcf2458 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:25.391539 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75738429-0f82-46f7-b2be-ecddabcf2458 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:25.391846 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75738429-0f82-46f7-b2be-ecddabcf2458 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:25.393250 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75738429-0f82-46f7-b2be-ecddabcf2458 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:25.401223 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 621/1264] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:05:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:05:25.407677 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419de711-213d-4455-b3f0-9e74b2b676be None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:25.475806 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:25.508920 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419de711-213d-4455-b3f0-9e74b2b676be None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:25.545314 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151fdf5-2a3e-4d60-90ef-3e681f67368d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:25.546044 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151fdf5-2a3e-4d60-90ef-3e681f67368d None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:25.546333 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151fdf5-2a3e-4d60-90ef-3e681f67368d None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:25.546613 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151fdf5-2a3e-4d60-90ef-3e681f67368d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:25.557787 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d151fdf5-2a3e-4d60-90ef-3e681f67368d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:25.559283 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d151fdf5-2a3e-4d60-90ef-3e681f67368d None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:25.573634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-d151fdf5-2a3e-4d60-90ef-3e681f67368d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:05:25.573634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:05:25.573634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:05:25.573634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:05:25.573634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:05:25.573634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:05:25.573634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:05:25.573634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:05:25.573634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:05:25.573634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:05:25.573634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:05:25.573634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:05:25.573634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:05:25.573634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:05:25.573634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:05:25.573634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:05:25.574765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:05:25.574765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:05:25.574765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:05:25.574765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:05:25.574765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:05:25.574765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:05:25.574765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:05:25.574765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:05:25.574765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:05:25.574765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:05:25.574765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:05:25.574765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:05:25.574765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:05:25.574765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:05:25.574765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:05:25.574765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:05:25.574765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:05:25.575928 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:05:25.575928 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:05:25.575928 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:05:25.575928 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 644/1265] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:05:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 18 12:05:25.580675 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75738429-0f82-46f7-b2be-ecddabcf2458 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:25.668152 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:25.686896 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75738429-0f82-46f7-b2be-ecddabcf2458 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:25.719215 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f0ab4-07cf-4e1e-ad45-2074711446cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:25.720380 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f0ab4-07cf-4e1e-ad45-2074711446cb None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:25.720692 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f0ab4-07cf-4e1e-ad45-2074711446cb None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:25.720936 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f0ab4-07cf-4e1e-ad45-2074711446cb None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:25.721580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f0ab4-07cf-4e1e-ad45-2074711446cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:25.721885 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f0ab4-07cf-4e1e-ad45-2074711446cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:25.723248 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f0ab4-07cf-4e1e-ad45-2074711446cb None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:25.735990 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 622/1266] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:05:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:05:25.741720 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151fdf5-2a3e-4d60-90ef-3e681f67368d None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:25.842049 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151fdf5-2a3e-4d60-90ef-3e681f67368d None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:25.862259 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:25.876287 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597cc59-7282-407d-9970-e703442d3fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:25.876920 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597cc59-7282-407d-9970-e703442d3fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:25.877206 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597cc59-7282-407d-9970-e703442d3fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:25.877482 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597cc59-7282-407d-9970-e703442d3fb4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:25.885069 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c597cc59-7282-407d-9970-e703442d3fb4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:25.886516 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c597cc59-7282-407d-9970-e703442d3fb4 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:25.898069 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-c597cc59-7282-407d-9970-e703442d3fb4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 18 12:05:25.898069 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:05:25.898069 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:05:25.898069 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:05:25.898069 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:05:25.898069 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:05:25.898069 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:05:25.898069 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:05:25.898069 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:05:25.898069 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:05:25.898069 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:05:25.898069 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:05:25.898069 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 12:05:25.898069 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 12:05:25.898069 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 12:05:25.898069 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 12:05:25.903233 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:05:25.903233 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:05:25.903233 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:05:25.903233 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:05:25.903233 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:05:25.903233 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:05:25.903233 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:05:25.903233 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:05:25.903233 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:05:25.903233 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:05:25.903233 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:05:25.903233 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:05:25.903233 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:05:25.903233 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:05:25.903233 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:05:25.903233 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:05:25.903233 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:05:25.903233 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:05:25.904483 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:05:25.904483 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:05:25.904483 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 12:05:25.904483 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 12:05:25.904483 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 12:05:25.904483 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 12:05:25.904483 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 12:05:25.904483 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 12:05:25.904483 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 18 12:05:25.904483 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:05:25.904483 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 645/1267] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:05:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:05:25.905278 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f0ab4-07cf-4e1e-ad45-2074711446cb None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:26.006637 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f0ab4-07cf-4e1e-ad45-2074711446cb None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:26.041509 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27293118-e755-4fb4-a902-55e530e1c8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:26.042103 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27293118-e755-4fb4-a902-55e530e1c8f8 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:26.042390 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27293118-e755-4fb4-a902-55e530e1c8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:26.042658 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27293118-e755-4fb4-a902-55e530e1c8f8 None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:26.043383 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27293118-e755-4fb4-a902-55e530e1c8f8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:26.043734 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27293118-e755-4fb4-a902-55e530e1c8f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:26.045149 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27293118-e755-4fb4-a902-55e530e1c8f8 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:26.053841 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:26.062134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 623/1268] 10.222.0.98 () {56 vars in 1143 bytes} [Thu Aug 18 12:05:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:05:26.067922 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597cc59-7282-407d-9970-e703442d3fb4 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:26.167344 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597cc59-7282-407d-9970-e703442d3fb4 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:26.200845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa47e482-05f2-4faf-a287-f16100ad4c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:26.201463 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47e482-05f2-4faf-a287-f16100ad4c92 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:26.201747 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47e482-05f2-4faf-a287-f16100ad4c92 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:26.202021 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47e482-05f2-4faf-a287-f16100ad4c92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:26.213363 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa47e482-05f2-4faf-a287-f16100ad4c92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:26.214816 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa47e482-05f2-4faf-a287-f16100ad4c92 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:26.228769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-aa47e482-05f2-4faf-a287-f16100ad4c92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:05:26.228769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:05:26.228769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:05:26.228769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:05:26.228769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:05:26.228769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:05:26.228769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:05:26.228769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:05:26.228769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:05:26.228769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:05:26.228769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:05:26.228769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:05:26.228769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:05:26.228769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:05:26.228769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:05:26.228769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:05:26.230032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:05:26.230032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:05:26.230032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:05:26.230032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:05:26.230032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:05:26.230032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:05:26.230032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:05:26.230032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:05:26.230032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:05:26.230032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:05:26.230032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:05:26.230032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:05:26.230032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:05:26.230032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:05:26.230032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:05:26.230032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:05:26.230032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:05:26.231180 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:05:26.231180 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:05:26.231180 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:05:26.231180 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 646/1269] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:05:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:05:26.235844 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27293118-e755-4fb4-a902-55e530e1c8f8 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:26.335389 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27293118-e755-4fb4-a902-55e530e1c8f8 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:26.369881 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873d1e1-263b-45fb-8ecf-b935e550b24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:26.370561 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873d1e1-263b-45fb-8ecf-b935e550b24f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:26.370845 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873d1e1-263b-45fb-8ecf-b935e550b24f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:26.371123 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873d1e1-263b-45fb-8ecf-b935e550b24f None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:26.371836 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8873d1e1-263b-45fb-8ecf-b935e550b24f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:26.372163 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8873d1e1-263b-45fb-8ecf-b935e550b24f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:26.373566 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8873d1e1-263b-45fb-8ecf-b935e550b24f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:26.385549 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 624/1270] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:05:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:05:26.391275 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa47e482-05f2-4faf-a287-f16100ad4c92 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:26.423903 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:26.492932 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa47e482-05f2-4faf-a287-f16100ad4c92 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:26.528025 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66637739-186f-4b59-b655-ad6ca3fbae62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:26.528654 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66637739-186f-4b59-b655-ad6ca3fbae62 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:26.528940 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66637739-186f-4b59-b655-ad6ca3fbae62 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:26.529217 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66637739-186f-4b59-b655-ad6ca3fbae62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:26.536080 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66637739-186f-4b59-b655-ad6ca3fbae62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:26.537515 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66637739-186f-4b59-b655-ad6ca3fbae62 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:26.549638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-66637739-186f-4b59-b655-ad6ca3fbae62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:05:26.549638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:05:26.549638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:05:26.549638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:05:26.549638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:05:26.549638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:05:26.549638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:05:26.549638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:05:26.549638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:05:26.549638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:05:26.549638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:05:26.549638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:05:26.549638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 12:05:26.549638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 12:05:26.549638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 12:05:26.552840 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 12:05:26.552840 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:05:26.552840 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:05:26.552840 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:05:26.552840 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:05:26.552840 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:05:26.552840 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:05:26.552840 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:05:26.552840 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:05:26.552840 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:05:26.552840 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:05:26.552840 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:05:26.552840 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:05:26.552840 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:05:26.552840 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:05:26.552840 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:05:26.552840 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:05:26.553743 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 12:05:26.553743 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 12:05:26.553743 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:05:26.553743 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:05:26.553743 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:05:26.553743 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:05:26.553743 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:05:26.553743 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:05:26.553743 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 647/1271] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:05:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:05:26.556804 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873d1e1-263b-45fb-8ecf-b935e550b24f None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:26.615606 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:26.657245 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873d1e1-263b-45fb-8ecf-b935e550b24f None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:26.691619 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333768bb-c171-43c1-a0f0-dae5f40e9fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:26.694720 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333768bb-c171-43c1-a0f0-dae5f40e9fdb None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:26.694720 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333768bb-c171-43c1-a0f0-dae5f40e9fdb None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:26.694720 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333768bb-c171-43c1-a0f0-dae5f40e9fdb None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:26.694720 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333768bb-c171-43c1-a0f0-dae5f40e9fdb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:26.694720 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333768bb-c171-43c1-a0f0-dae5f40e9fdb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:26.695724 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333768bb-c171-43c1-a0f0-dae5f40e9fdb None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:26.710780 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 625/1272] 10.222.0.98 () {56 vars in 1148 bytes} [Thu Aug 18 12:05:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:05:26.713962 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66637739-186f-4b59-b655-ad6ca3fbae62 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:26.808338 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:26.816836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66637739-186f-4b59-b655-ad6ca3fbae62 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:26.861431 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b46ba9-f151-4ef0-ada4-83e3a3f14a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:26.862118 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b46ba9-f151-4ef0-ada4-83e3a3f14a39 None admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:26.862402 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b46ba9-f151-4ef0-ada4-83e3a3f14a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:26.862685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b46ba9-f151-4ef0-ada4-83e3a3f14a39 None admin] PATH_INFO: `/v3/projects/6f676a5169464d83af4267db25025728/users/193637acadc14ec4a48ab79566ba3dab/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:26.864743 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b46ba9-f151-4ef0-ada4-83e3a3f14a39 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f676a5169464d83af4267db25025728, user_id=193637acadc14ec4a48ab79566ba3dab, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:26.866324 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b46ba9-f151-4ef0-ada4-83e3a3f14a39 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:26.966932 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 648/1273] 10.222.0.98 () {58 vars in 1439 bytes} [Thu Aug 18 12:05:26 2022] PUT /identity/v3/projects/6f676a5169464d83af4267db25025728/users/193637acadc14ec4a48ab79566ba3dab/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:05:28.858272 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a383a-e0fb-45fd-98e2-23addd007a16 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:28.858797 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a383a-e0fb-45fd-98e2-23addd007a16 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:28.858931 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a383a-e0fb-45fd-98e2-23addd007a16 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:28.860196 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 626/1274] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 12:05:28.865653 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf4874-79db-4d2b-9ea3-573417801967 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:28.866138 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf4874-79db-4d2b-9ea3-573417801967 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:28.866265 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf4874-79db-4d2b-9ea3-573417801967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:28.890338 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-90cf4874-79db-4d2b-9ea3-573417801967 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:05:28.891575 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-90cf4874-79db-4d2b-9ea3-573417801967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:05:28.959569 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 649/1275] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:05:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:05:29.234238 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a383a-e0fb-45fd-98e2-23addd007a16 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:29.334987 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a383a-e0fb-45fd-98e2-23addd007a16 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:29.358953 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:29.375458 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b93f1c-b5e3-4fea-9b57-d384b01a7957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:29.376083 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b93f1c-b5e3-4fea-9b57-d384b01a7957 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:29.376912 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b93f1c-b5e3-4fea-9b57-d384b01a7957 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:29.377189 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b93f1c-b5e3-4fea-9b57-d384b01a7957 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:29.387060 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b93f1c-b5e3-4fea-9b57-d384b01a7957 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 18 12:05:29.392434 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-35b93f1c-b5e3-4fea-9b57-d384b01a7957 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:05:29.392434 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:05:29.392434 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:05:29.392434 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:05:29.392434 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:05:29.392434 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:05:29.392434 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:05:29.392434 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:05:29.392434 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:05:29.392434 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:05:29.392434 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:05:29.392434 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:05:29.392434 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 18 12:05:29.392434 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 18 12:05:29.392434 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 18 12:05:29.392434 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 18 12:05:29.393593 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 18 12:05:29.393593 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 18 12:05:29.393593 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 18 12:05:29.393593 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 18 12:05:29.393593 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:05:29.393593 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:05:29.393593 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:05:29.393593 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:05:29.393593 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:05:29.393593 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:05:29.393593 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:05:29.393593 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:05:29.393593 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:05:29.393593 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:05:29.393593 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:05:29.393593 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:05:29.394637 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:05:29.394637 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:05:29.394637 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:05:29.394637 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:05:29.394637 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 18 12:05:29.394637 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 18 12:05:29.394637 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 18 12:05:29.394637 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 18 12:05:29.394637 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 18 12:05:29.394637 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 18 12:05:29.394637 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 18 12:05:29.394637 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:05:29.394637 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 627/1276] 10.222.0.98 () {56 vars in 1096 bytes} [Thu Aug 18 12:05:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:05:29.399815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf4874-79db-4d2b-9ea3-573417801967 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:29.481623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:29.499557 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf4874-79db-4d2b-9ea3-573417801967 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:29.534566 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ed081-0484-49ac-b160-3594841c9ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:29.535172 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ed081-0484-49ac-b160-3594841c9ab3 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:29.535470 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ed081-0484-49ac-b160-3594841c9ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:29.535744 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ed081-0484-49ac-b160-3594841c9ab3 None admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:29.536405 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07ed081-0484-49ac-b160-3594841c9ab3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:29.536711 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07ed081-0484-49ac-b160-3594841c9ab3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:29.538221 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07ed081-0484-49ac-b160-3594841c9ab3 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:29.546954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 650/1277] 10.222.0.98 () {56 vars in 1099 bytes} [Thu Aug 18 12:05:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:05:29.553496 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b93f1c-b5e3-4fea-9b57-d384b01a7957 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:29.663372 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b93f1c-b5e3-4fea-9b57-d384b01a7957 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:29.698367 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d634df30-4e31-40f1-8dde-3b132cf94b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:29.701772 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634df30-4e31-40f1-8dde-3b132cf94b8b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:29.701772 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634df30-4e31-40f1-8dde-3b132cf94b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:29.701772 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634df30-4e31-40f1-8dde-3b132cf94b8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:29.709764 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d634df30-4e31-40f1-8dde-3b132cf94b8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:29.711197 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d634df30-4e31-40f1-8dde-3b132cf94b8b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:29.725525 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-d634df30-4e31-40f1-8dde-3b132cf94b8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:05:29.725525 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:05:29.725525 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:05:29.725525 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:05:29.725525 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:05:29.725525 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:05:29.725525 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:05:29.725525 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:05:29.725525 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:05:29.725525 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:05:29.725525 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:05:29.725525 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:05:29.725525 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:05:29.725525 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:05:29.725525 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:05:29.725525 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:05:29.726837 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:05:29.726837 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:05:29.726837 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:05:29.726837 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:05:29.726837 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:05:29.726837 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:05:29.726837 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:05:29.726837 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:05:29.726837 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:05:29.726837 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:05:29.726837 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:05:29.726837 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:05:29.726837 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:05:29.726837 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:05:29.726837 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:05:29.726837 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:05:29.726837 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:05:29.727796 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:05:29.727796 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:05:29.727796 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:05:29.727796 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 628/1278] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:05:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:05:29.732662 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ed081-0484-49ac-b160-3594841c9ab3 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:29.835593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ed081-0484-49ac-b160-3594841c9ab3 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:29.870481 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f922e5-f9c9-4707-8c54-163923bd1ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:29.871209 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f922e5-f9c9-4707-8c54-163923bd1ff3 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:29.871361 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f922e5-f9c9-4707-8c54-163923bd1ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:29.871668 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f922e5-f9c9-4707-8c54-163923bd1ff3 None admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:29.872313 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f922e5-f9c9-4707-8c54-163923bd1ff3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:29.872624 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f922e5-f9c9-4707-8c54-163923bd1ff3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:29.873986 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f922e5-f9c9-4707-8c54-163923bd1ff3 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:29.886322 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 651/1279] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:05:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:05:29.892285 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d634df30-4e31-40f1-8dde-3b132cf94b8b None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:29.924577 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:29.993701 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d634df30-4e31-40f1-8dde-3b132cf94b8b None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:30.028544 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed68d12-3d18-4725-92e3-59e4b0b7fba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:30.029172 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed68d12-3d18-4725-92e3-59e4b0b7fba4 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:30.029457 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed68d12-3d18-4725-92e3-59e4b0b7fba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:30.029730 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed68d12-3d18-4725-92e3-59e4b0b7fba4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:30.037703 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed68d12-3d18-4725-92e3-59e4b0b7fba4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:30.039215 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed68d12-3d18-4725-92e3-59e4b0b7fba4 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:30.050921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-9ed68d12-3d18-4725-92e3-59e4b0b7fba4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 18 12:05:30.050921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:05:30.050921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:05:30.050921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:05:30.050921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:05:30.050921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:05:30.050921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:05:30.050921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:05:30.050921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:05:30.050921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:05:30.050921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:05:30.050921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:05:30.050921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 18 12:05:30.050921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 18 12:05:30.050921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 18 12:05:30.050921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 18 12:05:30.052020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:05:30.052020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:05:30.052020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 18 12:05:30.052020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:05:30.052020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 18 12:05:30.052020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 18 12:05:30.052020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:05:30.052020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:05:30.052020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:05:30.052020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:05:30.052020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:05:30.052020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:05:30.052020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:05:30.052020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:05:30.052020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:05:30.052020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:05:30.052020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:05:30.052020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:05:30.052954 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:05:30.052954 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:05:30.052954 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 18 12:05:30.052954 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 18 12:05:30.052954 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 18 12:05:30.052954 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 18 12:05:30.052954 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 18 12:05:30.052954 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 18 12:05:30.052954 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 18 12:05:30.052954 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:05:30.052954 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 629/1280] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:05:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:05:30.059771 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f922e5-f9c9-4707-8c54-163923bd1ff3 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:30.063567 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:30.158260 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f922e5-f9c9-4707-8c54-163923bd1ff3 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:30.191981 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff05127-a733-4f03-93ed-6fbe9aa9fbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:30.192578 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff05127-a733-4f03-93ed-6fbe9aa9fbe0 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:30.192865 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff05127-a733-4f03-93ed-6fbe9aa9fbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:30.193142 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff05127-a733-4f03-93ed-6fbe9aa9fbe0 None admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:30.193862 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff05127-a733-4f03-93ed-6fbe9aa9fbe0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:30.194173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff05127-a733-4f03-93ed-6fbe9aa9fbe0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:30.195628 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff05127-a733-4f03-93ed-6fbe9aa9fbe0 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:30.212487 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 652/1281] 10.222.0.98 () {56 vars in 1143 bytes} [Thu Aug 18 12:05:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:05:30.218473 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed68d12-3d18-4725-92e3-59e4b0b7fba4 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:30.309426 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:30.318711 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed68d12-3d18-4725-92e3-59e4b0b7fba4 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:30.351910 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c24c184-211a-4d72-95ed-607680db04cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:30.352513 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c24c184-211a-4d72-95ed-607680db04cd None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:30.352802 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c24c184-211a-4d72-95ed-607680db04cd None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:30.353074 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c24c184-211a-4d72-95ed-607680db04cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:30.424471 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c24c184-211a-4d72-95ed-607680db04cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:30.425848 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c24c184-211a-4d72-95ed-607680db04cd None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:30.514696 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:30.528548 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 630/1282] 10.222.0.98 () {58 vars in 1282 bytes} [Thu Aug 18 12:05:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 18 12:05:30.535096 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff05127-a733-4f03-93ed-6fbe9aa9fbe0 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:30.599962 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:30.637352 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff05127-a733-4f03-93ed-6fbe9aa9fbe0 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:30.671565 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e91bb-6728-4831-8823-14ff31884d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:30.672195 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e91bb-6728-4831-8823-14ff31884d5e None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:30.672479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e91bb-6728-4831-8823-14ff31884d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:30.672753 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e91bb-6728-4831-8823-14ff31884d5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:30.685386 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e91bb-6728-4831-8823-14ff31884d5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:30.685386 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e91bb-6728-4831-8823-14ff31884d5e None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:30.699321 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-476e91bb-6728-4831-8823-14ff31884d5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:05:30.699321 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:05:30.699321 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:05:30.699321 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:05:30.699321 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:05:30.699321 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:05:30.699321 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:05:30.699321 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:05:30.699321 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:05:30.699321 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:05:30.699321 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:05:30.699321 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:05:30.699321 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 18 12:05:30.699321 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 18 12:05:30.699321 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 18 12:05:30.699321 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 18 12:05:30.700489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:05:30.700489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:05:30.700489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:05:30.700489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:05:30.700489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:05:30.700489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:05:30.700489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:05:30.700489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:05:30.700489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:05:30.700489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:05:30.700489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:05:30.700489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:05:30.700489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:05:30.700489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:05:30.700489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:05:30.700489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:05:30.700489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 18 12:05:30.701618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 18 12:05:30.701618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 18 12:05:30.701618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:05:30.701618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 653/1283] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:05:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:05:30.706839 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c24c184-211a-4d72-95ed-607680db04cd None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:30.805956 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c24c184-211a-4d72-95ed-607680db04cd None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:30.849830 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5dc750-299f-4aaa-9be6-a0b92de1126a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:30.850436 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5dc750-299f-4aaa-9be6-a0b92de1126a None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:30.850719 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5dc750-299f-4aaa-9be6-a0b92de1126a None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:30.851095 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5dc750-299f-4aaa-9be6-a0b92de1126a None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:30.851703 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5dc750-299f-4aaa-9be6-a0b92de1126a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:30.852025 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5dc750-299f-4aaa-9be6-a0b92de1126a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:30.853449 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5dc750-299f-4aaa-9be6-a0b92de1126a None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:30.866120 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 631/1284] 10.222.0.98 () {56 vars in 1109 bytes} [Thu Aug 18 12:05:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:05:30.871828 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e91bb-6728-4831-8823-14ff31884d5e None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:30.973217 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e91bb-6728-4831-8823-14ff31884d5e None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:30.988204 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:31.008412 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e660de4-3b0c-4e75-a835-07498154f5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:31.009042 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e660de4-3b0c-4e75-a835-07498154f5e0 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:31.009324 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e660de4-3b0c-4e75-a835-07498154f5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:31.009598 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e660de4-3b0c-4e75-a835-07498154f5e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:31.016162 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e660de4-3b0c-4e75-a835-07498154f5e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:31.018195 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e660de4-3b0c-4e75-a835-07498154f5e0 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:31.029171 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application [None req-3e660de4-3b0c-4e75-a835-07498154f5e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:05:31.029171 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:05:31.029171 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:05:31.029171 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:05:31.029171 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:05:31.029171 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:05:31.029171 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:05:31.029171 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:05:31.029171 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:05:31.029171 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:05:31.029171 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:05:31.029171 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:05:31.029171 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 18 12:05:31.029171 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 18 12:05:31.029171 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 18 12:05:31.030479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 18 12:05:31.030479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:05:31.030479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:05:31.030479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:05:31.030479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:05:31.030479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:05:31.030479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:05:31.030479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:05:31.030479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application with Lock( Aug 18 12:05:31.030479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:05:31.030479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:05:31.030479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:05:31.030479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:05:31.030479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:05:31.030479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:05:31.030479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:05:31.030479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:05:31.031381 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 18 12:05:31.031381 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 18 12:05:31.031381 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 18 12:05:31.031381 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 18 12:05:31.031381 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 18 12:05:31.031381 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 18 12:05:31.031381 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 18 12:05:31.031381 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: ERROR keystone.server.flask.application Aug 18 12:05:31.031381 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 654/1285] 10.222.0.98 () {56 vars in 1111 bytes} [Thu Aug 18 12:05:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:05:31.036408 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5dc750-299f-4aaa-9be6-a0b92de1126a None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:31.088403 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:31.137829 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5dc750-299f-4aaa-9be6-a0b92de1126a None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:31.186424 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c547140c-ddd2-4162-86d7-326a8dd54288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:31.187039 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c547140c-ddd2-4162-86d7-326a8dd54288 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:31.187323 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c547140c-ddd2-4162-86d7-326a8dd54288 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:31.187632 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c547140c-ddd2-4162-86d7-326a8dd54288 None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:31.188330 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c547140c-ddd2-4162-86d7-326a8dd54288 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:31.188641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c547140c-ddd2-4162-86d7-326a8dd54288 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:31.190625 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c547140c-ddd2-4162-86d7-326a8dd54288 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:31.204623 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 632/1286] 10.222.0.98 () {56 vars in 1148 bytes} [Thu Aug 18 12:05:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:05:31.211377 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e660de4-3b0c-4e75-a835-07498154f5e0 None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:31.314774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e660de4-3b0c-4e75-a835-07498154f5e0 None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:31.347422 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49c9367-5dee-4713-a0f6-b3421fecbb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:31.348041 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c9367-5dee-4713-a0f6-b3421fecbb66 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:31.348438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c9367-5dee-4713-a0f6-b3421fecbb66 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:31.348635 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c9367-5dee-4713-a0f6-b3421fecbb66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:31.349354 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49c9367-5dee-4713-a0f6-b3421fecbb66 None admin] RBAC: Adding query filter params (role.id=c53a0e13b58642f291c856efe1f0b87d, scope.project.id=6f676a5169464d83af4267db25025728, user.id=193637acadc14ec4a48ab79566ba3dab) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:31.349666 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49c9367-5dee-4713-a0f6-b3421fecbb66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:31.351138 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49c9367-5dee-4713-a0f6-b3421fecbb66 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:31.356942 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 655/1287] 10.222.0.98 () {56 vars in 1374 bytes} [Thu Aug 18 12:05:31 2022] GET /identity/v3/role_assignments?user.id=193637acadc14ec4a48ab79566ba3dab&scope.project.id=6f676a5169464d83af4267db25025728&role.id=c53a0e13b58642f291c856efe1f0b87d => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:05:33.281758 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697010e-b18b-42a5-be88-3a60543477b7 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:33.282363 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697010e-b18b-42a5-be88-3a60543477b7 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:33.282363 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697010e-b18b-42a5-be88-3a60543477b7 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:33.284243 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 633/1288] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:05:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:05:33.290327 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51231e-6f75-42d1-914c-4ab93e37b2cb None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:33.290891 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51231e-6f75-42d1-914c-4ab93e37b2cb None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:33.290891 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51231e-6f75-42d1-914c-4ab93e37b2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:33.310626 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-8d51231e-6f75-42d1-914c-4ab93e37b2cb None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:05:33.311886 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-8d51231e-6f75-42d1-914c-4ab93e37b2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:05:33.346617 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:33.396615 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 656/1289] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:05:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:05:33.668333 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d697010e-b18b-42a5-be88-3a60543477b7 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:33.774693 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d697010e-b18b-42a5-be88-3a60543477b7 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:33.785389 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:33.808364 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a28618-c13e-4b3f-9ede-7abf01ebadd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:33.808986 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a28618-c13e-4b3f-9ede-7abf01ebadd1 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:33.809267 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a28618-c13e-4b3f-9ede-7abf01ebadd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:33.809540 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a28618-c13e-4b3f-9ede-7abf01ebadd1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:33.810117 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a28618-c13e-4b3f-9ede-7abf01ebadd1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:33.811531 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a28618-c13e-4b3f-9ede-7abf01ebadd1 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:33.820580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-b7a28618-c13e-4b3f-9ede-7abf01ebadd1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 12:05:33.820580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:05:33.820580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 12:05:33.820580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 12:05:33.820580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 12:05:33.820580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 12:05:33.820580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 12:05:33.820580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:05:33.820580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 12:05:33.820580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:05:33.820580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 12:05:33.820580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:05:33.820580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:05:33.820580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:05:33.820580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:05:33.820580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:05:33.820580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:05:33.821928 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:05:33.821928 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:05:33.821928 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:05:33.821928 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:05:33.821928 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:05:33.821928 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:05:33.821928 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 12:05:33.821928 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 12:05:33.821928 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 12:05:33.821928 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 12:05:33.821928 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:05:33.821928 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:05:33.821928 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:05:33.821928 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:05:33.821928 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:05:33.821928 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:05:33.826873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:05:33.826873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:05:33.826873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:05:33.826873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:05:33.826873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:05:33.826873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:05:33.826873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:05:33.826873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:05:33.826873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:05:33.826873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:05:33.826873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 12:05:33.826873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:05:33.826873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 12:05:33.826873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:05:33.826873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 634/1290] 10.222.0.98 () {56 vars in 1117 bytes} [Thu Aug 18 12:05:33 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:05:33.830720 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d51231e-6f75-42d1-914c-4ab93e37b2cb None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:33.925879 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d51231e-6f75-42d1-914c-4ab93e37b2cb None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:33.960077 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de699fcd-f5d7-47f0-bfe4-9d5d993f7c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:33.968731 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de699fcd-f5d7-47f0-bfe4-9d5d993f7c80 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:33.968731 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de699fcd-f5d7-47f0-bfe4-9d5d993f7c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:33.968731 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de699fcd-f5d7-47f0-bfe4-9d5d993f7c80 None admin] PATH_INFO: `/v3/services` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:33.968731 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de699fcd-f5d7-47f0-bfe4-9d5d993f7c80 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:33.968731 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de699fcd-f5d7-47f0-bfe4-9d5d993f7c80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:33.968731 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de699fcd-f5d7-47f0-bfe4-9d5d993f7c80 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:33.969673 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 657/1291] 10.222.0.98 () {56 vars in 1116 bytes} [Thu Aug 18 12:05:33 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:05:33.974132 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a28618-c13e-4b3f-9ede-7abf01ebadd1 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:34.077016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a28618-c13e-4b3f-9ede-7abf01ebadd1 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:34.109871 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd77717-798c-4a2b-9d68-b4a5086b09fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:34.110470 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd77717-798c-4a2b-9d68-b4a5086b09fc None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:34.110754 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd77717-798c-4a2b-9d68-b4a5086b09fc None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:34.111588 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd77717-798c-4a2b-9d68-b4a5086b09fc None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:34.112219 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd77717-798c-4a2b-9d68-b4a5086b09fc None admin] RBAC: Adding query filter params (type=placement) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:34.112531 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd77717-798c-4a2b-9d68-b4a5086b09fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:34.113889 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd77717-798c-4a2b-9d68-b4a5086b09fc None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:34.118309 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 635/1292] 10.222.0.98 () {56 vars in 1116 bytes} [Thu Aug 18 12:05:33 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:05:36.025264 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cdb8e-43da-42bf-877d-8ca5ed8563e1 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:36.025794 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cdb8e-43da-42bf-877d-8ca5ed8563e1 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:36.025888 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cdb8e-43da-42bf-877d-8ca5ed8563e1 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:36.027159 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 658/1293] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:05:36.032442 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7a8a7-dd58-49ec-b035-99060e0cd5df None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:36.032894 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7a8a7-dd58-49ec-b035-99060e0cd5df None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:36.033077 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7a8a7-dd58-49ec-b035-99060e0cd5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:36.051659 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-91f7a8a7-dd58-49ec-b035-99060e0cd5df None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:05:36.052866 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-91f7a8a7-dd58-49ec-b035-99060e0cd5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:05:36.083313 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:36.119547 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 636/1294] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:05:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:05:36.396155 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348cdb8e-43da-42bf-877d-8ca5ed8563e1 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:36.427959 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:36.499529 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348cdb8e-43da-42bf-877d-8ca5ed8563e1 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:36.533030 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa235f-5960-44fe-b34e-9f760c376f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:36.533636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa235f-5960-44fe-b34e-9f760c376f2d None admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:36.533906 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa235f-5960-44fe-b34e-9f760c376f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:36.534174 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa235f-5960-44fe-b34e-9f760c376f2d None admin] PATH_INFO: `/v3/services` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:36.534810 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aa235f-5960-44fe-b34e-9f760c376f2d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:36.536204 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aa235f-5960-44fe-b34e-9f760c376f2d None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:36.591149 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 659/1295] 10.222.0.98 () {60 vars in 1141 bytes} [Thu Aug 18 12:05:36 2022] POST /identity/v3/services => generated 246 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:05:38.511432 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790b4a9-9b71-409d-ab3d-4fb343470255 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:38.511989 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790b4a9-9b71-409d-ab3d-4fb343470255 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:38.511989 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790b4a9-9b71-409d-ab3d-4fb343470255 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:38.513370 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 637/1296] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:05:38.519060 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef9791-3963-4c6a-b9a7-e90b459f9a54 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:38.519511 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef9791-3963-4c6a-b9a7-e90b459f9a54 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:38.519635 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef9791-3963-4c6a-b9a7-e90b459f9a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:38.540253 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-22ef9791-3963-4c6a-b9a7-e90b459f9a54 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:05:38.541444 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-22ef9791-3963-4c6a-b9a7-e90b459f9a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:05:38.612907 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 660/1297] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:05:38 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:05:38.876165 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790b4a9-9b71-409d-ab3d-4fb343470255 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:38.979932 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790b4a9-9b71-409d-ab3d-4fb343470255 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:39.014751 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22625104-b901-4fd6-b033-098aac2a7fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:39.015382 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22625104-b901-4fd6-b033-098aac2a7fcd None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:39.015696 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22625104-b901-4fd6-b033-098aac2a7fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:39.015973 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22625104-b901-4fd6-b033-098aac2a7fcd None admin] PATH_INFO: `/v3/services/placement` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:39.016559 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22625104-b901-4fd6-b033-098aac2a7fcd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:39.017991 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22625104-b901-4fd6-b033-098aac2a7fcd None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:39.026802 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-22625104-b901-4fd6-b033-098aac2a7fcd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 12:05:39.026802 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:05:39.026802 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 12:05:39.026802 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 12:05:39.026802 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 12:05:39.026802 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 12:05:39.026802 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 12:05:39.026802 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:05:39.026802 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 12:05:39.026802 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:05:39.026802 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 12:05:39.026802 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:05:39.026802 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:05:39.026802 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:05:39.026802 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:05:39.026802 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:05:39.026802 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:05:39.028134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:05:39.028134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:05:39.028134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:05:39.028134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:05:39.028134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:05:39.028134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:05:39.028134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 12:05:39.028134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 12:05:39.028134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 12:05:39.028134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 12:05:39.028134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:05:39.028134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:05:39.028134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:05:39.028134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:05:39.028134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:05:39.028134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:05:39.029498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:05:39.029498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:05:39.029498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:05:39.029498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:05:39.029498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:05:39.029498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:05:39.029498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:05:39.029498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:05:39.029498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:05:39.029498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:05:39.029498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 12:05:39.029498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:05:39.029498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 12:05:39.029498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:05:39.029498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 638/1298] 10.222.0.98 () {56 vars in 1117 bytes} [Thu Aug 18 12:05:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:05:39.034019 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef9791-3963-4c6a-b9a7-e90b459f9a54 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:39.054166 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:39.132542 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef9791-3963-4c6a-b9a7-e90b459f9a54 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:39.165242 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b530a10-0c18-4b27-991a-2cd6829ba20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:39.165862 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b530a10-0c18-4b27-991a-2cd6829ba20e None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:39.166116 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b530a10-0c18-4b27-991a-2cd6829ba20e None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:39.166387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b530a10-0c18-4b27-991a-2cd6829ba20e None admin] PATH_INFO: `/v3/services` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:39.167269 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b530a10-0c18-4b27-991a-2cd6829ba20e None admin] RBAC: Adding query filter params (name=placement) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:39.167606 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b530a10-0c18-4b27-991a-2cd6829ba20e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:39.168965 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b530a10-0c18-4b27-991a-2cd6829ba20e None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:39.173930 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 661/1299] 10.222.0.98 () {56 vars in 1116 bytes} [Thu Aug 18 12:05:39 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:05:39.181007 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22625104-b901-4fd6-b033-098aac2a7fcd None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:39.213136 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:39.278721 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22625104-b901-4fd6-b033-098aac2a7fcd None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:39.311220 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a6c9da-cdc0-41ea-a8c3-0136a57482ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:39.311844 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6c9da-cdc0-41ea-a8c3-0136a57482ec None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:39.312123 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6c9da-cdc0-41ea-a8c3-0136a57482ec None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:39.312394 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6c9da-cdc0-41ea-a8c3-0136a57482ec None admin] PATH_INFO: `/v3/endpoints` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:39.313082 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a6c9da-cdc0-41ea-a8c3-0136a57482ec None admin] RBAC: Adding query filter params (interface=public, service_id=323dfe44ec414de4bbac95e4847aa2b0, region_id=RegionOne) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:39.313387 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a6c9da-cdc0-41ea-a8c3-0136a57482ec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:39.314817 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a6c9da-cdc0-41ea-a8c3-0136a57482ec None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:39.319664 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 639/1300] 10.222.0.98 () {56 vars in 1251 bytes} [Thu Aug 18 12:05:39 2022] GET /identity/v3/endpoints?service_id=323dfe44ec414de4bbac95e4847aa2b0&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:05:39.325417 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b530a10-0c18-4b27-991a-2cd6829ba20e None admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:39.409946 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:39.427825 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b530a10-0c18-4b27-991a-2cd6829ba20e None admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:39.470105 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbc927f-19a1-443e-ab26-6f9698f69760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:39.470715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc927f-19a1-443e-ab26-6f9698f69760 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:39.471002 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc927f-19a1-443e-ab26-6f9698f69760 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:39.471282 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc927f-19a1-443e-ab26-6f9698f69760 None admin] PATH_INFO: `/v3/services` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:39.471967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbc927f-19a1-443e-ab26-6f9698f69760 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:39.473965 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbc927f-19a1-443e-ab26-6f9698f69760 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:39.480927 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 662/1301] 10.222.0.98 () {56 vars in 1087 bytes} [Thu Aug 18 12:05:39 2022] GET /identity/v3/services => generated 2191 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:05:41.400727 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b3ae8-7938-4af1-88bb-a172dfd8cb87 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:41.401293 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b3ae8-7938-4af1-88bb-a172dfd8cb87 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:41.401293 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b3ae8-7938-4af1-88bb-a172dfd8cb87 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:41.402607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 640/1302] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:05:41.407956 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b465d-1a80-4414-89c1-8c3bc9f258af None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:41.408391 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b465d-1a80-4414-89c1-8c3bc9f258af None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:41.408523 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b465d-1a80-4414-89c1-8c3bc9f258af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:41.426806 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-a49b465d-1a80-4414-89c1-8c3bc9f258af None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:05:41.430534 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-a49b465d-1a80-4414-89c1-8c3bc9f258af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:05:41.500100 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 663/1303] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:05:41 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 18 12:05:41.776814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b3ae8-7938-4af1-88bb-a172dfd8cb87 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:41.872234 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:41.879507 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b3ae8-7938-4af1-88bb-a172dfd8cb87 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:41.913096 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32492930-5f55-42d3-b62d-fac5d923a5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:41.913722 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32492930-5f55-42d3-b62d-fac5d923a5c7 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:41.914007 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32492930-5f55-42d3-b62d-fac5d923a5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:41.914283 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32492930-5f55-42d3-b62d-fac5d923a5c7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:41.914866 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32492930-5f55-42d3-b62d-fac5d923a5c7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:41.916301 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32492930-5f55-42d3-b62d-fac5d923a5c7 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:41.926034 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-32492930-5f55-42d3-b62d-fac5d923a5c7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 12:05:41.926034 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:05:41.926034 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 18 12:05:41.926034 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 18 12:05:41.926034 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 18 12:05:41.926034 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 18 12:05:41.926034 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 18 12:05:41.926034 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:05:41.926034 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 12:05:41.926034 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:05:41.926034 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 18 12:05:41.926034 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:05:41.926034 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 18 12:05:41.926034 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 18 12:05:41.926034 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 18 12:05:41.926034 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 18 12:05:41.926034 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 18 12:05:41.933173 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 18 12:05:41.933173 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 18 12:05:41.933173 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 18 12:05:41.933173 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 18 12:05:41.933173 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 18 12:05:41.933173 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 18 12:05:41.933173 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 18 12:05:41.933173 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 18 12:05:41.933173 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 18 12:05:41.933173 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 18 12:05:41.933173 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 18 12:05:41.933173 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 18 12:05:41.933173 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 18 12:05:41.933173 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 18 12:05:41.933173 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 18 12:05:41.933173 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Aug 18 12:05:41.934151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 18 12:05:41.934151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Aug 18 12:05:41.934151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 18 12:05:41.934151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Aug 18 12:05:41.934151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 18 12:05:41.934151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 18 12:05:41.934151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 18 12:05:41.934151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Aug 18 12:05:41.934151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 18 12:05:41.934151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Aug 18 12:05:41.934151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 18 12:05:41.934151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 18 12:05:41.934151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 18 12:05:41.934151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Aug 18 12:05:41.934151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 641/1304] 10.222.0.98 () {56 vars in 1117 bytes} [Thu Aug 18 12:05:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 18 12:05:41.935038 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b465d-1a80-4414-89c1-8c3bc9f258af None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:42.016783 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:42.040141 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b465d-1a80-4414-89c1-8c3bc9f258af None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:42.075031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6133c995-c8c8-49a8-8a75-8c88b17c8a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:42.075743 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133c995-c8c8-49a8-8a75-8c88b17c8a21 None admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:42.076030 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133c995-c8c8-49a8-8a75-8c88b17c8a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:42.076324 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133c995-c8c8-49a8-8a75-8c88b17c8a21 None admin] PATH_INFO: `/v3/services` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:42.076951 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6133c995-c8c8-49a8-8a75-8c88b17c8a21 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:05:42.077259 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6133c995-c8c8-49a8-8a75-8c88b17c8a21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:42.078653 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6133c995-c8c8-49a8-8a75-8c88b17c8a21 None admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:42.084439 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 664/1305] 10.222.0.98 () {56 vars in 1116 bytes} [Thu Aug 18 12:05:41 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:05:42.090187 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32492930-5f55-42d3-b62d-fac5d923a5c7 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:05:42.195383 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32492930-5f55-42d3-b62d-fac5d923a5c7 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:05:42.228654 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4cda95-2bf4-4473-b486-68380916e73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:05:42.229254 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4cda95-2bf4-4473-b486-68380916e73c None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:05:42.229581 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4cda95-2bf4-4473-b486-68380916e73c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:05:42.229860 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4cda95-2bf4-4473-b486-68380916e73c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:05:42.230562 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4cda95-2bf4-4473-b486-68380916e73c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:05:42.231931 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4cda95-2bf4-4473-b486-68380916e73c None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:05:42.341315 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 642/1306] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:05:42 2022] POST /identity/v3/endpoints => generated 331 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:05:52.105382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=30 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:05:52.335731 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:12:06.865758 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9f23a-1dbf-41c8-a0a9-a70a84e8f905 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:06.866342 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9f23a-1dbf-41c8-a0a9-a70a84e8f905 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:06.866342 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9f23a-1dbf-41c8-a0a9-a70a84e8f905 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:06.873588 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5455a75-f174-41c4-a499-dfa555fdc645 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:06.875074 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5455a75-f174-41c4-a499-dfa555fdc645 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:06.875074 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5455a75-f174-41c4-a499-dfa555fdc645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:06.879111 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 665/1307] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:12:06 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:12:06.964929 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-b5455a75-f174-41c4-a499-dfa555fdc645 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:12:06.966123 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-b5455a75-f174-41c4-a499-dfa555fdc645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:07.099869 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 643/1308] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:12:07.396540 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b286e-f4f6-4fab-a7b3-623bed9b62c5 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:07.397087 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b286e-f4f6-4fab-a7b3-623bed9b62c5 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:07.397546 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b286e-f4f6-4fab-a7b3-623bed9b62c5 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:07.399376 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 666/1309] 10.222.0.98 () {54 vars in 965 bytes} [Thu Aug 18 12:12:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:12:07.407064 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4cb06-38f1-4450-91f7-7fef6bb7da97 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:07.407359 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4cb06-38f1-4450-91f7-7fef6bb7da97 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:07.407685 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4cb06-38f1-4450-91f7-7fef6bb7da97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:07.502717 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-03a4cb06-38f1-4450-91f7-7fef6bb7da97 None None] MFA Rules not processed for user `185644affb284c2b91e604cf99803ffb`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:12:07.504430 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-03a4cb06-38f1-4450-91f7-7fef6bb7da97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:07.527891 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:12:07.593015 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 644/1310] 10.222.0.98 () {58 vars in 1062 bytes} [Thu Aug 18 12:12:07 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:12:07.602076 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b286e-f4f6-4fab-a7b3-623bed9b62c5 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:12:07.603893 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-f17b286e-f4f6-4fab-a7b3-623bed9b62c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:07.732558 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b286e-f4f6-4fab-a7b3-623bed9b62c5 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:12:07.821225 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f97d8-b9cb-46a2-a4a0-3252309ef2b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:12:07.821829 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f97d8-b9cb-46a2-a4a0-3252309ef2b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:07.822118 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f97d8-b9cb-46a2-a4a0-3252309ef2b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:07.822390 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f97d8-b9cb-46a2-a4a0-3252309ef2b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:07.824128 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-133f97d8-b9cb-46a2-a4a0-3252309ef2b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:07.892848 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:12:07.962186 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133f97d8-b9cb-46a2-a4a0-3252309ef2b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:12:07.963878 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133f97d8-b9cb-46a2-a4a0-3252309ef2b1 service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:12:07.986071 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 667/1311] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:12:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:12:10.228510 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7417e8-36e8-47a1-aed9-907e15989a79 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:10.229086 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7417e8-36e8-47a1-aed9-907e15989a79 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:10.229086 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7417e8-36e8-47a1-aed9-907e15989a79 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:10.230922 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 645/1312] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:12:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:12:10.236291 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3f319-0122-4474-8113-130e30ab47ec None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:10.236681 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3f319-0122-4474-8113-130e30ab47ec None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:10.236856 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3f319-0122-4474-8113-130e30ab47ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:10.309349 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-15e3f319-0122-4474-8113-130e30ab47ec None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:12:10.310579 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-15e3f319-0122-4474-8113-130e30ab47ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:10.414936 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 668/1313] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:12:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:12:10.709484 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7417e8-36e8-47a1-aed9-907e15989a79 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:12:10.711156 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-9d7417e8-36e8-47a1-aed9-907e15989a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:10.822839 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7417e8-36e8-47a1-aed9-907e15989a79 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:12:10.896806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:12:10.899524 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6fb439-465b-4008-b30c-efc708b7d14f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:12:10.900139 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6fb439-465b-4008-b30c-efc708b7d14f service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:10.900422 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6fb439-465b-4008-b30c-efc708b7d14f service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:10.900701 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6fb439-465b-4008-b30c-efc708b7d14f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:10.902344 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-4a6fb439-465b-4008-b30c-efc708b7d14f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:11.011265 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6fb439-465b-4008-b30c-efc708b7d14f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:12:11.012728 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6fb439-465b-4008-b30c-efc708b7d14f service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:12:11.031051 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 646/1314] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:12:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:12:13.486947 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624dadf-5f29-4fd3-b4f4-3edab0891bc9 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:13.487508 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624dadf-5f29-4fd3-b4f4-3edab0891bc9 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:13.487508 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624dadf-5f29-4fd3-b4f4-3edab0891bc9 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:13.488866 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 669/1315] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:12:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:12:13.493932 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643657f-77c7-4c1a-b5e7-0dbf14707a07 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:13.494316 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643657f-77c7-4c1a-b5e7-0dbf14707a07 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:13.494496 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643657f-77c7-4c1a-b5e7-0dbf14707a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:13.562894 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-c643657f-77c7-4c1a-b5e7-0dbf14707a07 None None] MFA Rules not processed for user `31feea8cb0e3454194acf739392519e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:12:13.564161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-c643657f-77c7-4c1a-b5e7-0dbf14707a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:13.615732 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:12:13.663678 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 647/1316] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:12:13 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:12:13.949701 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a624dadf-5f29-4fd3-b4f4-3edab0891bc9 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:12:13.951282 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-a624dadf-5f29-4fd3-b4f4-3edab0891bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:13.961086 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:12:14.053230 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a624dadf-5f29-4fd3-b4f4-3edab0891bc9 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:12:14.125646 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5641c-5986-4891-bd1d-7c1673ffc9bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:12:14.126254 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5641c-5986-4891-bd1d-7c1673ffc9bc service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:14.126535 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5641c-5986-4891-bd1d-7c1673ffc9bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:14.126808 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5641c-5986-4891-bd1d-7c1673ffc9bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:14.128464 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-01e5641c-5986-4891-bd1d-7c1673ffc9bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:14.189635 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:12:14.235454 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e5641c-5986-4891-bd1d-7c1673ffc9bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:12:14.236871 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e5641c-5986-4891-bd1d-7c1673ffc9bc service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:12:14.254999 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 670/1317] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:12:13 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:12:17.061790 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b3776-3e0e-4a11-95f9-181f61735505 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:17.064984 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b3776-3e0e-4a11-95f9-181f61735505 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:17.064984 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b3776-3e0e-4a11-95f9-181f61735505 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:17.064984 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 648/1318] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:12:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:12:17.069067 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812ba69-0f75-4de1-a4da-1078e6aa21ae None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:17.069468 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812ba69-0f75-4de1-a4da-1078e6aa21ae None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:17.069631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812ba69-0f75-4de1-a4da-1078e6aa21ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:17.141570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-7812ba69-0f75-4de1-a4da-1078e6aa21ae None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:12:17.142805 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-7812ba69-0f75-4de1-a4da-1078e6aa21ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:17.262436 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 671/1319] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:12:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:12:17.582677 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accef667-a2d7-4033-8ca8-faf9dfee3349 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:17.582964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accef667-a2d7-4033-8ca8-faf9dfee3349 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:17.583265 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accef667-a2d7-4033-8ca8-faf9dfee3349 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:17.584643 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 649/1320] 10.222.0.98 () {54 vars in 965 bytes} [Thu Aug 18 12:12:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:12:17.590480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab3018-daa7-414d-ab6d-71e619846d01 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:17.590771 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab3018-daa7-414d-ab6d-71e619846d01 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:17.591052 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab3018-daa7-414d-ab6d-71e619846d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:17.612492 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:12:17.667289 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-2bab3018-daa7-414d-ab6d-71e619846d01 None None] MFA Rules not processed for user `185644affb284c2b91e604cf99803ffb`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:12:17.668542 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-2bab3018-daa7-414d-ab6d-71e619846d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:17.743293 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 672/1321] 10.222.0.98 () {58 vars in 1062 bytes} [Thu Aug 18 12:12:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:12:17.753070 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accef667-a2d7-4033-8ca8-faf9dfee3349 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:12:17.754599 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-accef667-a2d7-4033-8ca8-faf9dfee3349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:17.857936 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accef667-a2d7-4033-8ca8-faf9dfee3349 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:12:17.933641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ba9c4-4d68-4550-be38-ae68fe377699 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:12:17.934242 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ba9c4-4d68-4550-be38-ae68fe377699 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:17.934525 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ba9c4-4d68-4550-be38-ae68fe377699 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:17.934802 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ba9c4-4d68-4550-be38-ae68fe377699 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:17.936560 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-187ba9c4-4d68-4550-be38-ae68fe377699 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:17.944293 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:12:18.046677 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187ba9c4-4d68-4550-be38-ae68fe377699 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:12:18.048141 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187ba9c4-4d68-4550-be38-ae68fe377699 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:12:18.066980 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 650/1322] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:12:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:12:21.563196 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf1b4a-85d1-4f06-81ef-906818ed6e36 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:21.563892 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf1b4a-85d1-4f06-81ef-906818ed6e36 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:21.563892 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf1b4a-85d1-4f06-81ef-906818ed6e36 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:21.565180 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 673/1323] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:12:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:12:21.570592 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ba06f-35c0-498e-88b8-6ec715553deb None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:21.571065 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ba06f-35c0-498e-88b8-6ec715553deb None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:21.571178 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ba06f-35c0-498e-88b8-6ec715553deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:21.638367 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-ac9ba06f-35c0-498e-88b8-6ec715553deb None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:12:21.639603 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-ac9ba06f-35c0-498e-88b8-6ec715553deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:21.701399 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:12:21.741171 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 651/1324] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:12:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:12:22.027076 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf1b4a-85d1-4f06-81ef-906818ed6e36 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:12:22.028968 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-4edf1b4a-85d1-4f06-81ef-906818ed6e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:22.137877 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf1b4a-85d1-4f06-81ef-906818ed6e36 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:12:22.151632 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:12:22.214663 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3345733-952a-455b-ad5b-1238f66faf00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:12:22.215270 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3345733-952a-455b-ad5b-1238f66faf00 service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:22.215594 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3345733-952a-455b-ad5b-1238f66faf00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:22.215870 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3345733-952a-455b-ad5b-1238f66faf00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:22.217476 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-f3345733-952a-455b-ad5b-1238f66faf00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:22.329616 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3345733-952a-455b-ad5b-1238f66faf00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:12:22.330954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3345733-952a-455b-ad5b-1238f66faf00 service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:12:22.350059 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 674/1325] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:12:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:12:24.351142 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdac388-a229-44e8-8d92-fc08be4b9914 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:24.351712 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdac388-a229-44e8-8d92-fc08be4b9914 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:24.351712 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdac388-a229-44e8-8d92-fc08be4b9914 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:24.352992 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 652/1326] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:12:24.358344 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60817bc8-5bbb-4dda-b768-014cc5a0209a None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:24.358767 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60817bc8-5bbb-4dda-b768-014cc5a0209a None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:24.359046 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60817bc8-5bbb-4dda-b768-014cc5a0209a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:24.409261 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:12:24.427912 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-60817bc8-5bbb-4dda-b768-014cc5a0209a None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:12:24.429251 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-60817bc8-5bbb-4dda-b768-014cc5a0209a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:24.526698 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 675/1327] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:12:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:12:24.817565 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdac388-a229-44e8-8d92-fc08be4b9914 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:12:24.819194 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-3cdac388-a229-44e8-8d92-fc08be4b9914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:24.925781 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdac388-a229-44e8-8d92-fc08be4b9914 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:12:25.013185 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83972fe-2f5f-418b-a0ad-5662e8e46fde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:12:25.013808 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83972fe-2f5f-418b-a0ad-5662e8e46fde service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:25.014252 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83972fe-2f5f-418b-a0ad-5662e8e46fde service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:25.014378 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83972fe-2f5f-418b-a0ad-5662e8e46fde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:25.016026 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-b83972fe-2f5f-418b-a0ad-5662e8e46fde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:25.033620 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:12:25.131914 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83972fe-2f5f-418b-a0ad-5662e8e46fde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:12:25.134191 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83972fe-2f5f-418b-a0ad-5662e8e46fde service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:12:25.153449 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 653/1328] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:12:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:12:27.126530 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9fa770-19f7-4b10-a624-6b62f9d25a39 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:27.127074 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9fa770-19f7-4b10-a624-6b62f9d25a39 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:27.127074 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9fa770-19f7-4b10-a624-6b62f9d25a39 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:27.128468 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 676/1329] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:12:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:12:27.134631 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50387557-2f96-4aa7-8bd3-7ea88d4296f4 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:27.135130 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50387557-2f96-4aa7-8bd3-7ea88d4296f4 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:27.135222 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50387557-2f96-4aa7-8bd3-7ea88d4296f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:27.206296 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-50387557-2f96-4aa7-8bd3-7ea88d4296f4 None None] MFA Rules not processed for user `31feea8cb0e3454194acf739392519e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:12:27.207542 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-50387557-2f96-4aa7-8bd3-7ea88d4296f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:27.282301 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:12:27.315880 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 654/1330] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:12:27 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:12:27.685378 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9fa770-19f7-4b10-a624-6b62f9d25a39 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:12:27.687007 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-1e9fa770-19f7-4b10-a624-6b62f9d25a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:27.819022 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9fa770-19f7-4b10-a624-6b62f9d25a39 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:12:27.839621 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:12:27.894315 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ddc25-5a2b-4e72-ac6d-a6adc25978e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:12:27.894930 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ddc25-5a2b-4e72-ac6d-a6adc25978e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:27.895236 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ddc25-5a2b-4e72-ac6d-a6adc25978e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:27.895542 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ddc25-5a2b-4e72-ac6d-a6adc25978e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:27.897163 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-e23ddc25-5a2b-4e72-ac6d-a6adc25978e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:28.010125 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23ddc25-5a2b-4e72-ac6d-a6adc25978e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:12:28.011616 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23ddc25-5a2b-4e72-ac6d-a6adc25978e3 service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:12:28.030324 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 677/1331] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:12:27 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:12:30.375392 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ecb2c8-7eaf-4ad2-901d-be85e0aee19d None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:30.375954 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ecb2c8-7eaf-4ad2-901d-be85e0aee19d None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:30.375954 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ecb2c8-7eaf-4ad2-901d-be85e0aee19d None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:30.377227 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 655/1332] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:12:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:12:30.382593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68555dfd-9ecd-4787-86c2-a206fdd49d80 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:30.383597 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68555dfd-9ecd-4787-86c2-a206fdd49d80 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:30.383702 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68555dfd-9ecd-4787-86c2-a206fdd49d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:30.448150 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:12:30.450940 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-68555dfd-9ecd-4787-86c2-a206fdd49d80 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:12:30.452212 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-68555dfd-9ecd-4787-86c2-a206fdd49d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:30.549069 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 678/1333] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:12:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:12:30.840294 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ecb2c8-7eaf-4ad2-901d-be85e0aee19d None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:12:30.842097 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-c1ecb2c8-7eaf-4ad2-901d-be85e0aee19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:30.951179 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ecb2c8-7eaf-4ad2-901d-be85e0aee19d None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:12:31.025112 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b08cfd-9733-4751-8578-aa449ff23809 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:12:31.025725 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b08cfd-9733-4751-8578-aa449ff23809 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:31.026022 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b08cfd-9733-4751-8578-aa449ff23809 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:31.026300 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b08cfd-9733-4751-8578-aa449ff23809 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:31.028247 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-d1b08cfd-9733-4751-8578-aa449ff23809 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:31.055543 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:12:31.137425 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b08cfd-9733-4751-8578-aa449ff23809 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:12:31.138851 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b08cfd-9733-4751-8578-aa449ff23809 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:12:31.157475 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 656/1334] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:12:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:12:33.106588 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca97cec-9d87-48c0-985b-f68e6d93ecb2 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:33.107154 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca97cec-9d87-48c0-985b-f68e6d93ecb2 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:33.107154 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca97cec-9d87-48c0-985b-f68e6d93ecb2 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:33.108510 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 679/1335] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:12:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:12:33.113684 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d1dbbc-7906-48e8-bc2a-9d33a0a8e0b3 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:33.114130 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d1dbbc-7906-48e8-bc2a-9d33a0a8e0b3 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:33.114319 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d1dbbc-7906-48e8-bc2a-9d33a0a8e0b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:33.180341 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-62d1dbbc-7906-48e8-bc2a-9d33a0a8e0b3 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:12:33.181558 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-62d1dbbc-7906-48e8-bc2a-9d33a0a8e0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:33.254764 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:12:33.277200 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 657/1336] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:12:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:12:33.568446 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca97cec-9d87-48c0-985b-f68e6d93ecb2 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:12:33.570042 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-8ca97cec-9d87-48c0-985b-f68e6d93ecb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:33.680857 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca97cec-9d87-48c0-985b-f68e6d93ecb2 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:12:33.711276 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:12:33.755492 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f84d87-a067-48c4-8035-0b41bb9aaebc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:12:33.756104 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f84d87-a067-48c4-8035-0b41bb9aaebc service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:33.756421 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f84d87-a067-48c4-8035-0b41bb9aaebc service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:33.756705 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f84d87-a067-48c4-8035-0b41bb9aaebc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:33.758902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-89f84d87-a067-48c4-8035-0b41bb9aaebc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:33.867931 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f84d87-a067-48c4-8035-0b41bb9aaebc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:12:33.869284 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f84d87-a067-48c4-8035-0b41bb9aaebc service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:12:33.887528 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 680/1337] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:12:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:12:36.589247 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901fd699-0bdc-46c2-8c46-c5a621f2101c None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:36.589787 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901fd699-0bdc-46c2-8c46-c5a621f2101c None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:36.589787 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901fd699-0bdc-46c2-8c46-c5a621f2101c None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:36.591196 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 658/1338] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:12:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:12:36.596478 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394677ec-4a43-4055-bf96-64a38b4f3301 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:36.597038 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394677ec-4a43-4055-bf96-64a38b4f3301 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:36.597038 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394677ec-4a43-4055-bf96-64a38b4f3301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:36.666916 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-394677ec-4a43-4055-bf96-64a38b4f3301 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:12:36.668170 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-394677ec-4a43-4055-bf96-64a38b4f3301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:36.676628 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:12:36.767272 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 681/1339] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:12:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:12:37.051886 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901fd699-0bdc-46c2-8c46-c5a621f2101c None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:12:37.053433 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-901fd699-0bdc-46c2-8c46-c5a621f2101c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:37.162078 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901fd699-0bdc-46c2-8c46-c5a621f2101c None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:12:37.236350 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1fa518-4d40-47ec-a90a-a5a8f80410c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:12:37.236953 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1fa518-4d40-47ec-a90a-a5a8f80410c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:37.237230 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1fa518-4d40-47ec-a90a-a5a8f80410c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:37.237501 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1fa518-4d40-47ec-a90a-a5a8f80410c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:37.239089 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-8c1fa518-4d40-47ec-a90a-a5a8f80410c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:37.276062 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:12:37.350536 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1fa518-4d40-47ec-a90a-a5a8f80410c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:12:37.352438 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1fa518-4d40-47ec-a90a-a5a8f80410c7 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:12:37.371548 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 659/1340] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:12:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:12:41.654380 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae23191-840d-4466-b23e-99d6376099c4 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:41.654925 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae23191-840d-4466-b23e-99d6376099c4 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:41.654925 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae23191-840d-4466-b23e-99d6376099c4 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:41.656395 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 682/1341] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:12:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 12:12:41.661607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdea5aa-9c67-4c6a-a3cd-113f68b3d5ed None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:41.662029 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdea5aa-9c67-4c6a-a3cd-113f68b3d5ed None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:41.662174 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdea5aa-9c67-4c6a-a3cd-113f68b3d5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:41.737860 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-ccdea5aa-9c67-4c6a-a3cd-113f68b3d5ed None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:12:41.740743 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-ccdea5aa-9c67-4c6a-a3cd-113f68b3d5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:41.832935 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:12:41.844185 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 660/1342] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:12:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:12:42.133807 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae23191-840d-4466-b23e-99d6376099c4 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:12:42.135481 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-7ae23191-840d-4466-b23e-99d6376099c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:42.246236 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae23191-840d-4466-b23e-99d6376099c4 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:12:42.286882 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:12:42.322623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17e688-3a49-4ad6-baf3-ff1062d1972a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:12:42.323226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17e688-3a49-4ad6-baf3-ff1062d1972a service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:42.323548 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17e688-3a49-4ad6-baf3-ff1062d1972a service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:42.323813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17e688-3a49-4ad6-baf3-ff1062d1972a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:42.325424 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-ea17e688-3a49-4ad6-baf3-ff1062d1972a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:42.439515 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea17e688-3a49-4ad6-baf3-ff1062d1972a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:12:42.440880 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea17e688-3a49-4ad6-baf3-ff1062d1972a service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:12:42.461280 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 683/1343] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:12:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:12:45.442481 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d735c2e-2575-409c-995c-28566f267c53 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:45.443045 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d735c2e-2575-409c-995c-28566f267c53 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:45.443045 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d735c2e-2575-409c-995c-28566f267c53 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:45.445010 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 661/1344] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:12:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:12:45.450158 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f76ca1e-86f2-431a-96d8-c918bbfc940a None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:45.450616 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f76ca1e-86f2-431a-96d8-c918bbfc940a None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:45.450730 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f76ca1e-86f2-431a-96d8-c918bbfc940a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:45.516512 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-2f76ca1e-86f2-431a-96d8-c918bbfc940a None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:12:45.517731 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-2f76ca1e-86f2-431a-96d8-c918bbfc940a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:45.536272 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:12:45.614124 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 684/1345] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:12:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:12:45.894722 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d735c2e-2575-409c-995c-28566f267c53 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:12:45.896306 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-9d735c2e-2575-409c-995c-28566f267c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:46.002605 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d735c2e-2575-409c-995c-28566f267c53 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:12:46.083582 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9fd907-851a-47c0-ad85-51bd81dc3396 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:12:46.084284 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9fd907-851a-47c0-ad85-51bd81dc3396 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:46.084579 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9fd907-851a-47c0-ad85-51bd81dc3396 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:46.084859 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9fd907-851a-47c0-ad85-51bd81dc3396 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:46.086609 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-af9fd907-851a-47c0-ad85-51bd81dc3396 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:46.133535 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:12:46.202219 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9fd907-851a-47c0-ad85-51bd81dc3396 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:12:46.203943 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9fd907-851a-47c0-ad85-51bd81dc3396 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:12:46.222978 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 662/1346] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:12:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:12:50.361195 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0810cd0-2c2b-4e5f-8c22-b3dcc946b99f None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:50.361790 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0810cd0-2c2b-4e5f-8c22-b3dcc946b99f None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:50.361790 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0810cd0-2c2b-4e5f-8c22-b3dcc946b99f None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:50.364357 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 685/1347] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:12:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:12:50.369372 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ae91e-15e0-4db9-95f3-2f2b32b14f18 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:50.369888 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ae91e-15e0-4db9-95f3-2f2b32b14f18 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:50.369990 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ae91e-15e0-4db9-95f3-2f2b32b14f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:50.449061 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-7c1ae91e-15e0-4db9-95f3-2f2b32b14f18 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:12:50.450321 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-7c1ae91e-15e0-4db9-95f3-2f2b32b14f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:50.554221 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:12:50.557532 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 663/1348] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:12:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:12:50.841822 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0810cd0-2c2b-4e5f-8c22-b3dcc946b99f None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:12:50.843377 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-d0810cd0-2c2b-4e5f-8c22-b3dcc946b99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:50.972258 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0810cd0-2c2b-4e5f-8c22-b3dcc946b99f None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:12:51.025731 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:12:51.060066 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198133ff-6669-438a-b01e-3403af7cd9a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:12:51.060765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198133ff-6669-438a-b01e-3403af7cd9a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:51.061120 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198133ff-6669-438a-b01e-3403af7cd9a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:51.061463 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198133ff-6669-438a-b01e-3403af7cd9a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:51.063194 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-198133ff-6669-438a-b01e-3403af7cd9a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:51.179883 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198133ff-6669-438a-b01e-3403af7cd9a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:12:51.181355 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198133ff-6669-438a-b01e-3403af7cd9a8 service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:12:51.200518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 686/1349] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:12:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:12:56.129434 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41319f7-d826-4805-af76-f0de083f3e30 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:56.129936 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41319f7-d826-4805-af76-f0de083f3e30 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:56.130338 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41319f7-d826-4805-af76-f0de083f3e30 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:56.133128 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 664/1350] 10.222.0.98 () {54 vars in 921 bytes} [Thu Aug 18 12:12:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:12:56.137961 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064c9bb5-ae25-4c02-a092-2000276a293f None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:12:56.138790 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064c9bb5-ae25-4c02-a092-2000276a293f None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:12:56.139817 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064c9bb5-ae25-4c02-a092-2000276a293f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:12:56.242692 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-064c9bb5-ae25-4c02-a092-2000276a293f None None] MFA Rules not processed for user `193637acadc14ec4a48ab79566ba3dab`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:12:56.245330 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-064c9bb5-ae25-4c02-a092-2000276a293f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:12:56.283335 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:12:56.421765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 687/1351] 10.222.0.98 () {58 vars in 1018 bytes} [Thu Aug 18 12:12:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:13:00.787740 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeda5e0-57a1-4d77-b948-f6c2bef251b6 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:00.788606 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeda5e0-57a1-4d77-b948-f6c2bef251b6 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:00.789110 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeda5e0-57a1-4d77-b948-f6c2bef251b6 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:00.790634 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 665/1352] 10.222.0.98 () {54 vars in 921 bytes} [Thu Aug 18 12:13:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:13:00.795918 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d1b87-1510-4a88-ab47-a6f29d6384ba None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:00.796711 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d1b87-1510-4a88-ab47-a6f29d6384ba None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:00.797173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d1b87-1510-4a88-ab47-a6f29d6384ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:00.885914 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-e08d1b87-1510-4a88-ab47-a6f29d6384ba None None] MFA Rules not processed for user `193637acadc14ec4a48ab79566ba3dab`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:13:00.887164 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-e08d1b87-1510-4a88-ab47-a6f29d6384ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:00.975083 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:01.012385 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 688/1353] 10.222.0.98 () {58 vars in 1018 bytes} [Thu Aug 18 12:13:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:13:02.148106 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e0dcc-f95d-4ce1-8535-36b7e22e8922 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:02.148634 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e0dcc-f95d-4ce1-8535-36b7e22e8922 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:02.148634 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e0dcc-f95d-4ce1-8535-36b7e22e8922 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:02.149988 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 666/1354] 10.222.0.98 () {54 vars in 921 bytes} [Thu Aug 18 12:13:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:13:02.155206 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece46911-f6de-4226-9b56-2bdb82ec4010 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:02.155688 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece46911-f6de-4226-9b56-2bdb82ec4010 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:02.155810 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece46911-f6de-4226-9b56-2bdb82ec4010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:02.236945 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-ece46911-f6de-4226-9b56-2bdb82ec4010 None None] MFA Rules not processed for user `193637acadc14ec4a48ab79566ba3dab`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:13:02.238740 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-ece46911-f6de-4226-9b56-2bdb82ec4010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:02.345729 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 689/1355] 10.222.0.98 () {58 vars in 1018 bytes} [Thu Aug 18 12:13:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:13:05.747031 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac088fc9-ffeb-4b76-a746-875521fd7225 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:05.747521 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac088fc9-ffeb-4b76-a746-875521fd7225 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:05.747521 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac088fc9-ffeb-4b76-a746-875521fd7225 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:05.748866 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 667/1356] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:13:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:13:05.754212 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6699420-fa62-4d36-bd2c-342dc3342ce1 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:05.754639 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6699420-fa62-4d36-bd2c-342dc3342ce1 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:05.754768 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6699420-fa62-4d36-bd2c-342dc3342ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:05.803097 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:05.839956 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-b6699420-fa62-4d36-bd2c-342dc3342ce1 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:13:05.841186 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-b6699420-fa62-4d36-bd2c-342dc3342ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:05.962774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 690/1357] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:13:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:13:06.264652 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1082ce67-c304-4452-9aeb-1441bec87a74 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:06.271229 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1082ce67-c304-4452-9aeb-1441bec87a74 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:06.271229 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1082ce67-c304-4452-9aeb-1441bec87a74 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:06.271229 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 668/1358] 10.222.0.98 () {54 vars in 962 bytes} [Thu Aug 18 12:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:13:06.282832 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0ef76-f2bf-4c60-9f67-456b4a1575b0 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:06.283310 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0ef76-f2bf-4c60-9f67-456b4a1575b0 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:06.284654 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0ef76-f2bf-4c60-9f67-456b4a1575b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:06.373312 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-a9b0ef76-f2bf-4c60-9f67-456b4a1575b0 None None] MFA Rules not processed for user `171c72a6b29e418bbb4ea5c6faffe3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:13:06.379590 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-a9b0ef76-f2bf-4c60-9f67-456b4a1575b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:06.432274 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:06.504341 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 691/1359] 10.222.0.98 () {58 vars in 1059 bytes} [Thu Aug 18 12:13:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:13:06.512403 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1082ce67-c304-4452-9aeb-1441bec87a74 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:13:06.514125 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-1082ce67-c304-4452-9aeb-1441bec87a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:06.701335 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1082ce67-c304-4452-9aeb-1441bec87a74 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:13:06.784180 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:06.826806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b28dd-efde-4a71-93c6-883f9d8fa32c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:13:06.827443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b28dd-efde-4a71-93c6-883f9d8fa32c service nova] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:06.827734 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b28dd-efde-4a71-93c6-883f9d8fa32c service nova] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:06.828011 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b28dd-efde-4a71-93c6-883f9d8fa32c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:06.829617 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-f35b28dd-efde-4a71-93c6-883f9d8fa32c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:06.924154 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35b28dd-efde-4a71-93c6-883f9d8fa32c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:13:06.926204 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35b28dd-efde-4a71-93c6-883f9d8fa32c service nova] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:13:06.945763 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 669/1360] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:13:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:13:09.078351 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37570b93-724e-4db0-8182-14e1571e8142 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:09.078883 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37570b93-724e-4db0-8182-14e1571e8142 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:09.078883 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37570b93-724e-4db0-8182-14e1571e8142 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:09.080940 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 692/1361] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:13:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:13:09.086299 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccaea3c-fa18-49ff-b5ab-8c4f9ae14497 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:09.086756 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccaea3c-fa18-49ff-b5ab-8c4f9ae14497 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:09.086866 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccaea3c-fa18-49ff-b5ab-8c4f9ae14497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:09.162718 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-5ccaea3c-fa18-49ff-b5ab-8c4f9ae14497 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:13:09.164757 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-5ccaea3c-fa18-49ff-b5ab-8c4f9ae14497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:09.206222 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:09.279138 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 670/1362] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:13:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:13:09.570898 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37570b93-724e-4db0-8182-14e1571e8142 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:13:09.572471 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-37570b93-724e-4db0-8182-14e1571e8142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:09.764621 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37570b93-724e-4db0-8182-14e1571e8142 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:13:09.775622 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:09.888526 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd240f77-348f-4bd9-9e07-721db4dd935c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:13:09.889300 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd240f77-348f-4bd9-9e07-721db4dd935c service nova] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:09.889690 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd240f77-348f-4bd9-9e07-721db4dd935c service nova] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:09.890065 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd240f77-348f-4bd9-9e07-721db4dd935c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:09.891820 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-cd240f77-348f-4bd9-9e07-721db4dd935c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:09.993887 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd240f77-348f-4bd9-9e07-721db4dd935c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:13:09.995530 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd240f77-348f-4bd9-9e07-721db4dd935c service nova] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:13:10.016278 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 693/1363] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:13:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:13:12.472456 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969266cd-676f-4823-8858-7301818c7046 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:12.473122 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969266cd-676f-4823-8858-7301818c7046 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:12.473122 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969266cd-676f-4823-8858-7301818c7046 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:12.474584 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 671/1364] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:13:12.480128 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f34e5f-7cc2-4c34-b912-012c20224ca9 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:12.480754 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f34e5f-7cc2-4c34-b912-012c20224ca9 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:12.480839 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f34e5f-7cc2-4c34-b912-012c20224ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:12.496934 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:12.561666 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-89f34e5f-7cc2-4c34-b912-012c20224ca9 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:13:12.562906 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-89f34e5f-7cc2-4c34-b912-012c20224ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:12.673100 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 694/1365] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:13:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:13:12.966250 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639fd28e-cc60-4acb-8330-13153838e3a3 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:12.966602 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639fd28e-cc60-4acb-8330-13153838e3a3 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:12.966977 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639fd28e-cc60-4acb-8330-13153838e3a3 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:12.969287 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 672/1366] 10.222.0.98 () {54 vars in 962 bytes} [Thu Aug 18 12:13:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:13:12.975611 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f283c27-b774-4ad0-8df2-8ab6d7a9f508 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:12.984854 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f283c27-b774-4ad0-8df2-8ab6d7a9f508 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:12.984854 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f283c27-b774-4ad0-8df2-8ab6d7a9f508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:13.059940 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-4f283c27-b774-4ad0-8df2-8ab6d7a9f508 None None] MFA Rules not processed for user `171c72a6b29e418bbb4ea5c6faffe3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:13:13.061300 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-4f283c27-b774-4ad0-8df2-8ab6d7a9f508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:13.101138 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:13.193349 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 695/1367] 10.222.0.98 () {58 vars in 1059 bytes} [Thu Aug 18 12:13:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:13:13.202151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639fd28e-cc60-4acb-8330-13153838e3a3 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:13:13.205490 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-639fd28e-cc60-4acb-8330-13153838e3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:13.369686 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639fd28e-cc60-4acb-8330-13153838e3a3 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:13:13.393585 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:13.502553 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a57988-4f17-4713-81fc-a596f686d890 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:13:13.503218 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a57988-4f17-4713-81fc-a596f686d890 service nova] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:13.503661 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a57988-4f17-4713-81fc-a596f686d890 service nova] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:13.503747 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a57988-4f17-4713-81fc-a596f686d890 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:13.505339 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-20a57988-4f17-4713-81fc-a596f686d890 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:13.598230 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a57988-4f17-4713-81fc-a596f686d890 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:13:13.599871 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a57988-4f17-4713-81fc-a596f686d890 service nova] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:13:13.619272 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 673/1368] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:13:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:13:14.064751 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8476fbf-f4da-4d21-8926-879508b41d8e None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:14.065259 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8476fbf-f4da-4d21-8926-879508b41d8e None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:14.065259 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8476fbf-f4da-4d21-8926-879508b41d8e None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:14.066809 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 696/1369] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:13:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:13:14.071988 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619ba7b-ae67-4384-bdbc-fa5b62bc06ed None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:14.072278 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619ba7b-ae67-4384-bdbc-fa5b62bc06ed None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:14.072554 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619ba7b-ae67-4384-bdbc-fa5b62bc06ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:14.106312 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:14.145276 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-6619ba7b-ae67-4384-bdbc-fa5b62bc06ed None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:13:14.147782 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-6619ba7b-ae67-4384-bdbc-fa5b62bc06ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:14.255500 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 674/1370] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:13:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:13:14.847925 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61203a5-f8b9-4524-8fc5-e68c328dfc68 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:14.848188 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61203a5-f8b9-4524-8fc5-e68c328dfc68 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:14.848468 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61203a5-f8b9-4524-8fc5-e68c328dfc68 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:14.849976 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 697/1371] 10.222.0.98 () {54 vars in 964 bytes} [Thu Aug 18 12:13:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:13:14.854846 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35666f89-44b9-44a2-b2f2-7e1d5312793e None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:14.856231 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35666f89-44b9-44a2-b2f2-7e1d5312793e None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:14.856231 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35666f89-44b9-44a2-b2f2-7e1d5312793e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:14.940691 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-35666f89-44b9-44a2-b2f2-7e1d5312793e None None] MFA Rules not processed for user `c41b6fb3cb064a49b9236077a8faed54`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:13:14.942210 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-35666f89-44b9-44a2-b2f2-7e1d5312793e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:15.006150 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:15.038676 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 675/1372] 10.222.0.98 () {58 vars in 1061 bytes} [Thu Aug 18 12:13:14 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:13:15.047280 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61203a5-f8b9-4524-8fc5-e68c328dfc68 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:13:15.049361 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-c61203a5-f8b9-4524-8fc5-e68c328dfc68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:15.160126 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61203a5-f8b9-4524-8fc5-e68c328dfc68 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:13:15.203690 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:15.243705 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d282b2e-cb06-4879-8e94-d52771ca8a10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:13:15.244321 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d282b2e-cb06-4879-8e94-d52771ca8a10 service cinder] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:15.244600 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d282b2e-cb06-4879-8e94-d52771ca8a10 service cinder] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:15.244879 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d282b2e-cb06-4879-8e94-d52771ca8a10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:15.246485 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-5d282b2e-cb06-4879-8e94-d52771ca8a10 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:15.373685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d282b2e-cb06-4879-8e94-d52771ca8a10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:13:15.375181 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d282b2e-cb06-4879-8e94-d52771ca8a10 service cinder] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:13:15.396585 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 698/1373] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:13:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:13:15.772592 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c12fc-cd20-4944-952d-5c3393f20ed6 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:15.772877 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c12fc-cd20-4944-952d-5c3393f20ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:15.773156 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c12fc-cd20-4944-952d-5c3393f20ed6 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:15.774684 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 676/1374] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:13:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 12:13:15.779746 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a784e0d-cedd-464c-a499-8fabf26f361a None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:15.780036 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a784e0d-cedd-464c-a499-8fabf26f361a None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:15.780313 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a784e0d-cedd-464c-a499-8fabf26f361a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:15.848014 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-8a784e0d-cedd-464c-a499-8fabf26f361a None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:13:15.849285 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-8a784e0d-cedd-464c-a499-8fabf26f361a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:15.870214 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:15.946229 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 699/1375] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:13:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:13:16.235253 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c12fc-cd20-4944-952d-5c3393f20ed6 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:13:16.236958 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-066c12fc-cd20-4944-952d-5c3393f20ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:16.390208 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c12fc-cd20-4944-952d-5c3393f20ed6 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:13:16.452590 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:16.518670 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800d9482-45af-436a-811a-07fdf8cb5f2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:13:16.519839 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d9482-45af-436a-811a-07fdf8cb5f2d service nova] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:16.520291 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d9482-45af-436a-811a-07fdf8cb5f2d service nova] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:16.520577 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d9482-45af-436a-811a-07fdf8cb5f2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:16.522202 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-800d9482-45af-436a-811a-07fdf8cb5f2d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:16.614963 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800d9482-45af-436a-811a-07fdf8cb5f2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:13:16.616579 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800d9482-45af-436a-811a-07fdf8cb5f2d service nova] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:13:16.635384 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 677/1376] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:13:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:13:18.666657 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83536ab7-7826-4134-95f2-97ae0f853eba None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:18.668286 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83536ab7-7826-4134-95f2-97ae0f853eba None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:18.668286 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83536ab7-7826-4134-95f2-97ae0f853eba None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:18.669144 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 700/1377] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:13:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:13:18.674773 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb4929d-24f7-428e-9182-040239fe92af None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:18.675644 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb4929d-24f7-428e-9182-040239fe92af None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:18.676087 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb4929d-24f7-428e-9182-040239fe92af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:18.747341 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-4cb4929d-24f7-428e-9182-040239fe92af None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:13:18.748748 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-4cb4929d-24f7-428e-9182-040239fe92af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:18.755711 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:18.856946 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 678/1378] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:13:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:13:19.201165 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83536ab7-7826-4134-95f2-97ae0f853eba None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:13:19.202998 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-83536ab7-7826-4134-95f2-97ae0f853eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:19.398516 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83536ab7-7826-4134-95f2-97ae0f853eba None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:13:19.403352 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:19.546082 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303205a0-9553-4a47-9499-1770ea969db9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:13:19.546698 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303205a0-9553-4a47-9499-1770ea969db9 service nova] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:19.547041 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303205a0-9553-4a47-9499-1770ea969db9 service nova] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:19.547297 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303205a0-9553-4a47-9499-1770ea969db9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:19.549439 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-303205a0-9553-4a47-9499-1770ea969db9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:19.649069 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303205a0-9553-4a47-9499-1770ea969db9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:13:19.650585 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303205a0-9553-4a47-9499-1770ea969db9 service nova] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:13:19.679621 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 701/1379] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:13:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:13:21.793823 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a89bf-4bb9-48fa-a5a2-26c17eb50c50 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:21.794387 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a89bf-4bb9-48fa-a5a2-26c17eb50c50 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:21.794387 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a89bf-4bb9-48fa-a5a2-26c17eb50c50 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:21.795887 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 679/1380] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:13:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:13:21.801808 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7b34a-c00c-4e98-9b2b-21813e6dedfa None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:21.802281 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7b34a-c00c-4e98-9b2b-21813e6dedfa None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:21.802375 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7b34a-c00c-4e98-9b2b-21813e6dedfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:21.814484 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:21.872857 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-1da7b34a-c00c-4e98-9b2b-21813e6dedfa None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:13:21.874504 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-1da7b34a-c00c-4e98-9b2b-21813e6dedfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:21.975453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 702/1381] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:13:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:13:22.268553 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a89bf-4bb9-48fa-a5a2-26c17eb50c50 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:13:22.270743 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-5b3a89bf-4bb9-48fa-a5a2-26c17eb50c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:22.423184 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:22.434163 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a89bf-4bb9-48fa-a5a2-26c17eb50c50 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:13:22.555660 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4acac5-2f40-42d3-8569-df434346b5c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:13:22.556314 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4acac5-2f40-42d3-8569-df434346b5c4 service nova] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:22.556598 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4acac5-2f40-42d3-8569-df434346b5c4 service nova] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:22.556871 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4acac5-2f40-42d3-8569-df434346b5c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:22.558472 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-dc4acac5-2f40-42d3-8569-df434346b5c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:22.656321 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4acac5-2f40-42d3-8569-df434346b5c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:13:22.658870 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4acac5-2f40-42d3-8569-df434346b5c4 service nova] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:13:22.679122 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:22.683586 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 680/1382] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:13:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:13:24.961952 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b0a6f-3eb2-4a5f-99f3-63c998e76cfa None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:24.962698 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b0a6f-3eb2-4a5f-99f3-63c998e76cfa None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:24.962698 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b0a6f-3eb2-4a5f-99f3-63c998e76cfa None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:24.964607 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 703/1383] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:13:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:13:24.972801 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636cecc-4ee5-49a3-8448-5e6d0a61303c None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:24.973902 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636cecc-4ee5-49a3-8448-5e6d0a61303c None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:24.974359 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636cecc-4ee5-49a3-8448-5e6d0a61303c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:25.061905 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-6636cecc-4ee5-49a3-8448-5e6d0a61303c None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:13:25.065459 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-6636cecc-4ee5-49a3-8448-5e6d0a61303c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:25.213726 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 681/1384] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:13:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:13:25.646215 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b0a6f-3eb2-4a5f-99f3-63c998e76cfa None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:13:25.648144 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-b24b0a6f-3eb2-4a5f-99f3-63c998e76cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:25.757928 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:25.832000 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b0a6f-3eb2-4a5f-99f3-63c998e76cfa None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:13:25.969692 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea8021-8ff6-4c68-b647-79256093405f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:13:25.970780 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea8021-8ff6-4c68-b647-79256093405f service nova] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:25.970980 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea8021-8ff6-4c68-b647-79256093405f service nova] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:25.971140 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea8021-8ff6-4c68-b647-79256093405f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:25.973107 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-36ea8021-8ff6-4c68-b647-79256093405f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:26.033886 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:26.075574 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea8021-8ff6-4c68-b647-79256093405f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:13:26.077210 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea8021-8ff6-4c68-b647-79256093405f service nova] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:13:26.096819 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 704/1385] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:13:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:13:28.084168 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4068f84-7eb6-4946-8728-0f19fe2fc8c2 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:28.085009 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4068f84-7eb6-4946-8728-0f19fe2fc8c2 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:28.085201 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4068f84-7eb6-4946-8728-0f19fe2fc8c2 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:28.086727 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 682/1386] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:13:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:13:28.093296 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e14dd2-331d-48eb-b208-e14d8c6c6ce2 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:28.094660 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e14dd2-331d-48eb-b208-e14d8c6c6ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:28.094926 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e14dd2-331d-48eb-b208-e14d8c6c6ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:28.172351 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-49e14dd2-331d-48eb-b208-e14d8c6c6ce2 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:13:28.173654 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-49e14dd2-331d-48eb-b208-e14d8c6c6ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:28.276468 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 705/1387] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:13:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:13:28.595151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4068f84-7eb6-4946-8728-0f19fe2fc8c2 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:13:28.596742 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-d4068f84-7eb6-4946-8728-0f19fe2fc8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:28.678101 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:28.758240 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4068f84-7eb6-4946-8728-0f19fe2fc8c2 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:13:28.890922 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13684d-ae27-46c8-9d24-d16b8b418c73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:13:28.891706 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13684d-ae27-46c8-9d24-d16b8b418c73 service nova] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:28.892320 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13684d-ae27-46c8-9d24-d16b8b418c73 service nova] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:28.892606 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13684d-ae27-46c8-9d24-d16b8b418c73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:28.894504 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-9b13684d-ae27-46c8-9d24-d16b8b418c73 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:28.938215 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:28.994058 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b13684d-ae27-46c8-9d24-d16b8b418c73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:13:28.996132 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b13684d-ae27-46c8-9d24-d16b8b418c73 service nova] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:13:29.015651 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 683/1388] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:13:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:13:31.129560 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907b28f-7c78-4e2f-8194-e1707c7c2e3d None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:31.130138 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907b28f-7c78-4e2f-8194-e1707c7c2e3d None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:31.130138 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907b28f-7c78-4e2f-8194-e1707c7c2e3d None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:31.131649 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 706/1389] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:13:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 12:13:31.136778 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d08de-fdc0-4d15-b30c-ea1073e3062a None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:31.137192 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d08de-fdc0-4d15-b30c-ea1073e3062a None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:31.137338 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d08de-fdc0-4d15-b30c-ea1073e3062a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:31.208786 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-ad5d08de-fdc0-4d15-b30c-ea1073e3062a None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:13:31.210190 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-ad5d08de-fdc0-4d15-b30c-ea1073e3062a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:31.314992 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 684/1390] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:13:31.607799 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907b28f-7c78-4e2f-8194-e1707c7c2e3d None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:13:31.609408 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-c907b28f-7c78-4e2f-8194-e1707c7c2e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:31.642256 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:31.761045 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907b28f-7c78-4e2f-8194-e1707c7c2e3d None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:13:31.878117 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:31.891730 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6ee083-bade-4908-bc3a-739f8e202764 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:13:31.893046 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ee083-bade-4908-bc3a-739f8e202764 service nova] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:31.893046 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ee083-bade-4908-bc3a-739f8e202764 service nova] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:31.893267 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ee083-bade-4908-bc3a-739f8e202764 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:31.894792 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-3d6ee083-bade-4908-bc3a-739f8e202764 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:31.992794 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6ee083-bade-4908-bc3a-739f8e202764 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:13:31.994422 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6ee083-bade-4908-bc3a-739f8e202764 service nova] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:13:32.013928 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 707/1391] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:13:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:13:32.433981 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a0bf9-58d2-4cfb-b773-13c38c289117 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:32.434514 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a0bf9-58d2-4cfb-b773-13c38c289117 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:32.434514 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a0bf9-58d2-4cfb-b773-13c38c289117 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:32.435981 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 685/1392] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:13:32.441981 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f8567-158b-4908-95a5-ecb32330490e None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:32.442397 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f8567-158b-4908-95a5-ecb32330490e None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:32.442542 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f8567-158b-4908-95a5-ecb32330490e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:32.511958 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-498f8567-158b-4908-95a5-ecb32330490e None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:13:32.513206 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-498f8567-158b-4908-95a5-ecb32330490e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:32.573361 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:32.613025 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 708/1393] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:13:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:13:32.914435 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c52d74-ef16-446e-a180-e4e53ec89c33 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:32.914782 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c52d74-ef16-446e-a180-e4e53ec89c33 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:32.915132 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c52d74-ef16-446e-a180-e4e53ec89c33 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:32.916494 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 686/1394] 10.222.0.98 () {54 vars in 967 bytes} [Thu Aug 18 12:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:13:32.922302 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5b970-3995-423e-826e-18fd825d6d63 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:32.922623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5b970-3995-423e-826e-18fd825d6d63 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:32.922907 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5b970-3995-423e-826e-18fd825d6d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:32.997431 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-68e5b970-3995-423e-826e-18fd825d6d63 None None] MFA Rules not processed for user `d020063702b0481e9f4111680766f7fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:13:32.998636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-68e5b970-3995-423e-826e-18fd825d6d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:33.075608 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 709/1395] 10.222.0.98 () {58 vars in 1064 bytes} [Thu Aug 18 12:13:32 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:13:33.083718 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c52d74-ef16-446e-a180-e4e53ec89c33 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:13:33.085248 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-73c52d74-ef16-446e-a180-e4e53ec89c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:33.099592 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:33.194821 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c52d74-ef16-446e-a180-e4e53ec89c33 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:13:33.270071 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b417e43-cbcf-4a86-9b52-736cf8af27cb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd020063702b0481e9f4111680766f7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:13:33.270685 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b417e43-cbcf-4a86-9b52-736cf8af27cb service glance] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:33.270984 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b417e43-cbcf-4a86-9b52-736cf8af27cb service glance] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:33.271259 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b417e43-cbcf-4a86-9b52-736cf8af27cb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:33.272906 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-9b417e43-cbcf-4a86-9b52-736cf8af27cb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:33.344147 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:33.385300 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b417e43-cbcf-4a86-9b52-736cf8af27cb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:13:33.386754 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b417e43-cbcf-4a86-9b52-736cf8af27cb service glance] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:13:33.406465 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 687/1396] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:13:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:13:34.165086 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c27de-45be-4525-a6ea-dd607c631394 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:34.165598 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c27de-45be-4525-a6ea-dd607c631394 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:34.165598 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c27de-45be-4525-a6ea-dd607c631394 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:34.168904 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 710/1397] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:13:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:13:34.174956 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd440f2-0eb1-43f7-a152-e2d7977bb85d None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:34.175520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd440f2-0eb1-43f7-a152-e2d7977bb85d None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:34.175607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd440f2-0eb1-43f7-a152-e2d7977bb85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:34.242559 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-3dd440f2-0eb1-43f7-a152-e2d7977bb85d None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:13:34.243839 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-3dd440f2-0eb1-43f7-a152-e2d7977bb85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:34.277996 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee579fb-24a7-4841-a2d3-8a5dcd861662 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:34.278291 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee579fb-24a7-4841-a2d3-8a5dcd861662 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:34.278573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee579fb-24a7-4841-a2d3-8a5dcd861662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:34.346990 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-3ee579fb-24a7-4841-a2d3-8a5dcd861662 None None] MFA Rules not processed for user `b6880278f7304051b9ef03e927637e74`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:13:34.348257 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-3ee579fb-24a7-4841-a2d3-8a5dcd861662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:34.352941 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 688/1398] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:13:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:13:34.357183 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:34.442300 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 711/1399] 10.222.0.98 () {58 vars in 1009 bytes} [Thu Aug 18 12:13:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:13:34.488885 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d8621-bebb-4619-851e-c9e8bf3e49dc None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:34.489178 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d8621-bebb-4619-851e-c9e8bf3e49dc None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:34.489455 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d8621-bebb-4619-851e-c9e8bf3e49dc None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:34.490800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 689/1400] 10.222.0.98 () {54 vars in 943 bytes} [Thu Aug 18 12:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:13:34.496961 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b18b52-a428-424d-96ed-3e4f24752393 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:34.497337 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b18b52-a428-424d-96ed-3e4f24752393 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:34.497803 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b18b52-a428-424d-96ed-3e4f24752393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:34.575121 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-02b18b52-a428-424d-96ed-3e4f24752393 None None] MFA Rules not processed for user `771591fe31cb4e8c9c4320c2a7b2719d`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:13:34.576361 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-02b18b52-a428-424d-96ed-3e4f24752393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:34.638855 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d8621-bebb-4619-851e-c9e8bf3e49dc None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:13:34.640960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-093d8621-bebb-4619-851e-c9e8bf3e49dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:34.665168 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:34.666535 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:34.687984 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 712/1401] 10.222.0.98 () {58 vars in 1040 bytes} [Thu Aug 18 12:13:34 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:13:34.698102 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b18b52-a428-424d-96ed-3e4f24752393 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:13:34.708264 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b18b52-a428-424d-96ed-3e4f24752393 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:13:34.795930 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d8621-bebb-4619-851e-c9e8bf3e49dc None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:13:34.813439 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaba466-d87a-45fb-90f6-291e39f77b41 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771591fe31cb4e8c9c4320c2a7b2719d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:13:34.814052 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaba466-d87a-45fb-90f6-291e39f77b41 service swift] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:34.814337 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaba466-d87a-45fb-90f6-291e39f77b41 service swift] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:34.814614 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaba466-d87a-45fb-90f6-291e39f77b41 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:34.816304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-6eaba466-d87a-45fb-90f6-291e39f77b41 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:34.896564 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:34.902882 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaba466-d87a-45fb-90f6-291e39f77b41 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:13:34.904521 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaba466-d87a-45fb-90f6-291e39f77b41 service swift] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:13:34.913391 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 713/1402] 10.222.0.98 () {60 vars in 1384 bytes} [Thu Aug 18 12:13:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 12:13:34.916387 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c5b335-615a-4daa-82f3-e2131bee12bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:13:34.916994 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5b335-615a-4daa-82f3-e2131bee12bc service nova] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:34.917415 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5b335-615a-4daa-82f3-e2131bee12bc service nova] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:34.917582 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5b335-615a-4daa-82f3-e2131bee12bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:34.919140 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-52c5b335-615a-4daa-82f3-e2131bee12bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:35.012866 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c5b335-615a-4daa-82f3-e2131bee12bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:13:35.015132 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c5b335-615a-4daa-82f3-e2131bee12bc service nova] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:13:35.033533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 690/1403] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:13:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:13:37.311440 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66009868-861a-46c6-bea3-0130f71bddd4 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:37.312064 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66009868-861a-46c6-bea3-0130f71bddd4 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:37.312064 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66009868-861a-46c6-bea3-0130f71bddd4 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:37.313449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 714/1404] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:13:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:13:37.319059 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3916b3c-5427-45c9-9ce4-14df1c96e21c None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:37.319597 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3916b3c-5427-45c9-9ce4-14df1c96e21c None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:37.319687 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3916b3c-5427-45c9-9ce4-14df1c96e21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:37.403119 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-e3916b3c-5427-45c9-9ce4-14df1c96e21c None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:13:37.404388 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-e3916b3c-5427-45c9-9ce4-14df1c96e21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:37.471633 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:37.522752 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 691/1405] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:13:37.814128 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66009868-861a-46c6-bea3-0130f71bddd4 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:13:37.815713 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-66009868-861a-46c6-bea3-0130f71bddd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:37.827193 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:38.004251 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66009868-861a-46c6-bea3-0130f71bddd4 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:13:38.098931 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:38.138521 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc3ebd4-e561-4eb8-8a3f-e7388b35fdfe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:13:38.139148 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3ebd4-e561-4eb8-8a3f-e7388b35fdfe service nova] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:38.139453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3ebd4-e561-4eb8-8a3f-e7388b35fdfe service nova] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:38.139729 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3ebd4-e561-4eb8-8a3f-e7388b35fdfe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:38.141527 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-bdc3ebd4-e561-4eb8-8a3f-e7388b35fdfe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:38.242967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc3ebd4-e561-4eb8-8a3f-e7388b35fdfe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:13:38.244770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc3ebd4-e561-4eb8-8a3f-e7388b35fdfe service nova] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:13:38.264191 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 715/1406] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:13:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:13:40.310621 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fdb89e-c7d2-4787-aa0f-9c2592eb5650 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:40.311198 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fdb89e-c7d2-4787-aa0f-9c2592eb5650 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:40.311198 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fdb89e-c7d2-4787-aa0f-9c2592eb5650 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:40.312718 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 692/1407] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:13:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:13:40.317800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879fb8c-6319-44d4-a250-595614db1970 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:40.318241 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879fb8c-6319-44d4-a250-595614db1970 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:40.318401 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879fb8c-6319-44d4-a250-595614db1970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:40.391121 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-f879fb8c-6319-44d4-a250-595614db1970 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:13:40.392673 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-f879fb8c-6319-44d4-a250-595614db1970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:40.445114 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:40.509727 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 716/1408] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:13:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:13:40.804979 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fdb89e-c7d2-4787-aa0f-9c2592eb5650 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:13:40.806723 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-e5fdb89e-c7d2-4787-aa0f-9c2592eb5650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:40.972038 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fdb89e-c7d2-4787-aa0f-9c2592eb5650 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:13:41.020161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:41.106042 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a23eb35-5a7f-4f22-9316-3994fd8b3868 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:13:41.106757 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23eb35-5a7f-4f22-9316-3994fd8b3868 service nova] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:41.107134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23eb35-5a7f-4f22-9316-3994fd8b3868 service nova] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:41.107517 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23eb35-5a7f-4f22-9316-3994fd8b3868 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:41.109313 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-0a23eb35-5a7f-4f22-9316-3994fd8b3868 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:41.218059 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a23eb35-5a7f-4f22-9316-3994fd8b3868 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:13:41.219834 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a23eb35-5a7f-4f22-9316-3994fd8b3868 service nova] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:13:41.279029 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 693/1409] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:13:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:13:50.507640 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=23 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:51.274776 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=79 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:58.409570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dadd2f3-c244-4d40-a40e-1cfdd301302c None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:58.410194 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dadd2f3-c244-4d40-a40e-1cfdd301302c None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:58.410194 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dadd2f3-c244-4d40-a40e-1cfdd301302c None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:58.411474 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 717/1410] 10.222.0.98 () {54 vars in 918 bytes} [Thu Aug 18 12:13:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:13:58.416826 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b4ddd-c148-4817-b3c0-10890d5d675d None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:58.417265 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b4ddd-c148-4817-b3c0-10890d5d675d None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:58.417384 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b4ddd-c148-4817-b3c0-10890d5d675d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:58.495894 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-6f5b4ddd-c148-4817-b3c0-10890d5d675d None None] MFA Rules not processed for user `193637acadc14ec4a48ab79566ba3dab`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:13:58.497122 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-6f5b4ddd-c148-4817-b3c0-10890d5d675d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:58.608819 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 694/1411] 10.222.0.98 () {58 vars in 1015 bytes} [Thu Aug 18 12:13:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:13:58.623836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27029a5a-3bb5-48af-878e-9d5ef92fa964 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:58.624128 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27029a5a-3bb5-48af-878e-9d5ef92fa964 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:58.624410 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27029a5a-3bb5-48af-878e-9d5ef92fa964 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:58.625790 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 718/1412] 10.222.0.98 () {54 vars in 918 bytes} [Thu Aug 18 12:13:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:13:58.630713 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d658a-7f52-4f79-b8e5-7a8d62a6f6b1 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:13:58.631011 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d658a-7f52-4f79-b8e5-7a8d62a6f6b1 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:13:58.631292 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d658a-7f52-4f79-b8e5-7a8d62a6f6b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:13:58.706544 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:13:58.709883 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-a04d658a-7f52-4f79-b8e5-7a8d62a6f6b1 None None] MFA Rules not processed for user `171c72a6b29e418bbb4ea5c6faffe3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:13:58.711119 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-a04d658a-7f52-4f79-b8e5-7a8d62a6f6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:13:58.841097 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 695/1413] 10.222.0.98 () {58 vars in 1015 bytes} [Thu Aug 18 12:13:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:14:01.268259 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3ff60-f091-449a-bc94-84a222b05316 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:01.268775 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3ff60-f091-449a-bc94-84a222b05316 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:01.268775 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3ff60-f091-449a-bc94-84a222b05316 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:01.270168 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 719/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 12:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:14:01.275699 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce7dec-62e1-498d-9c32-7551b744e255 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:01.276116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce7dec-62e1-498d-9c32-7551b744e255 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:01.276256 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce7dec-62e1-498d-9c32-7551b744e255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:01.353956 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-adce7dec-62e1-498d-9c32-7551b744e255 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:14:01.355166 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-adce7dec-62e1-498d-9c32-7551b744e255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:01.412503 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:01.456778 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 696/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 12:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:14:01.746919 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f3ff60-f091-449a-bc94-84a222b05316 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:14:01.749098 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-72f3ff60-f091-449a-bc94-84a222b05316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:01.895564 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f3ff60-f091-449a-bc94-84a222b05316 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:14:01.969256 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:02.008418 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae477b8-81e7-4e68-bb61-0f3624d2f967 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:14:02.009073 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae477b8-81e7-4e68-bb61-0f3624d2f967 service nova] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:02.009565 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae477b8-81e7-4e68-bb61-0f3624d2f967 service nova] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:02.010080 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae477b8-81e7-4e68-bb61-0f3624d2f967 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:02.012117 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-5ae477b8-81e7-4e68-bb61-0f3624d2f967 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:02.102747 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae477b8-81e7-4e68-bb61-0f3624d2f967 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:14:02.105881 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae477b8-81e7-4e68-bb61-0f3624d2f967 service nova] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:14:02.123826 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 720/1416] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:14:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:14:03.016461 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36290b22-a754-4baf-b75f-76f8d9834cdd None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:03.017051 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36290b22-a754-4baf-b75f-76f8d9834cdd None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:03.017051 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36290b22-a754-4baf-b75f-76f8d9834cdd None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:03.018353 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 697/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 12:14:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:14:03.023358 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9a79c-ff12-4b65-a32f-3afa68f2f6e1 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:03.023867 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9a79c-ff12-4b65-a32f-3afa68f2f6e1 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:03.023971 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9a79c-ff12-4b65-a32f-3afa68f2f6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:03.100961 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-2da9a79c-ff12-4b65-a32f-3afa68f2f6e1 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:14:03.102169 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-2da9a79c-ff12-4b65-a32f-3afa68f2f6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:03.128534 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:03.197202 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 721/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 12:14:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:14:03.352398 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36290b22-a754-4baf-b75f-76f8d9834cdd None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:14:03.353846 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-36290b22-a754-4baf-b75f-76f8d9834cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:03.504383 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36290b22-a754-4baf-b75f-76f8d9834cdd None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:14:03.544152 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:03.615387 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c71555-7aad-4e5d-8fe3-a04535f83456 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:14:03.615936 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c71555-7aad-4e5d-8fe3-a04535f83456 service nova] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:03.616217 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c71555-7aad-4e5d-8fe3-a04535f83456 service nova] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:03.616491 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c71555-7aad-4e5d-8fe3-a04535f83456 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:03.618082 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-b1c71555-7aad-4e5d-8fe3-a04535f83456 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:03.711773 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c71555-7aad-4e5d-8fe3-a04535f83456 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:14:03.713397 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c71555-7aad-4e5d-8fe3-a04535f83456 service nova] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:14:03.731342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 698/1419] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:14:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:14:04.609915 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d19e7-2afe-45ed-b948-f2bc7f0f6e2a None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:04.610468 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d19e7-2afe-45ed-b948-f2bc7f0f6e2a None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:04.610468 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d19e7-2afe-45ed-b948-f2bc7f0f6e2a None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:04.611874 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 722/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 12:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:14:04.617036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02daaa-f867-470e-b80e-f0e5202e1ba6 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:04.617477 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02daaa-f867-470e-b80e-f0e5202e1ba6 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:04.617602 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02daaa-f867-470e-b80e-f0e5202e1ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:04.674201 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:04.696815 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-2d02daaa-f867-470e-b80e-f0e5202e1ba6 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:14:04.698069 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-2d02daaa-f867-470e-b80e-f0e5202e1ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:04.793985 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 699/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 12:14:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:14:04.951633 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d19e7-2afe-45ed-b948-f2bc7f0f6e2a None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:14:04.953254 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-328d19e7-2afe-45ed-b948-f2bc7f0f6e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:05.099556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d19e7-2afe-45ed-b948-f2bc7f0f6e2a None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:14:05.112035 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:05.214873 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec14df23-03e6-4eb7-a197-1b7266a3c6eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:14:05.215529 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14df23-03e6-4eb7-a197-1b7266a3c6eb service nova] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:05.215815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14df23-03e6-4eb7-a197-1b7266a3c6eb service nova] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:05.216091 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14df23-03e6-4eb7-a197-1b7266a3c6eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:05.217705 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-ec14df23-03e6-4eb7-a197-1b7266a3c6eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:05.306673 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec14df23-03e6-4eb7-a197-1b7266a3c6eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:14:05.308332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec14df23-03e6-4eb7-a197-1b7266a3c6eb service nova] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:14:05.329429 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 723/1422] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:14:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:14:06.202705 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7886161-d52f-4679-a040-babd8d712de9 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:06.203239 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7886161-d52f-4679-a040-babd8d712de9 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:06.203357 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7886161-d52f-4679-a040-babd8d712de9 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:06.204642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 700/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 12:14:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:14:06.209464 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66db017-2af0-4a6a-a9d3-327e52e5224d None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:06.209891 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66db017-2af0-4a6a-a9d3-327e52e5224d None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:06.210020 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66db017-2af0-4a6a-a9d3-327e52e5224d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:06.230360 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:06.287152 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-b66db017-2af0-4a6a-a9d3-327e52e5224d None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:14:06.288455 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-b66db017-2af0-4a6a-a9d3-327e52e5224d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:06.384803 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 724/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 12:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:14:06.540779 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7886161-d52f-4679-a040-babd8d712de9 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:14:06.542316 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-e7886161-d52f-4679-a040-babd8d712de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:06.652883 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:06.688522 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7886161-d52f-4679-a040-babd8d712de9 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:14:06.802689 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0f9445-7eab-42f9-9fca-2776341daa17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:14:06.803309 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0f9445-7eab-42f9-9fca-2776341daa17 service nova] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:06.803627 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0f9445-7eab-42f9-9fca-2776341daa17 service nova] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:06.803909 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0f9445-7eab-42f9-9fca-2776341daa17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:06.805547 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-5c0f9445-7eab-42f9-9fca-2776341daa17 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:06.882260 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:06.895653 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0f9445-7eab-42f9-9fca-2776341daa17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:14:06.897569 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0f9445-7eab-42f9-9fca-2776341daa17 service nova] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:14:06.915510 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 701/1425] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:14:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:14:07.777056 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41153563-3e62-4369-8814-d61189e32694 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:07.777612 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41153563-3e62-4369-8814-d61189e32694 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:07.777612 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41153563-3e62-4369-8814-d61189e32694 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:07.779001 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 725/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 12:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:14:07.783944 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73db71-b570-4e12-8397-eef41002e44f None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:07.784341 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73db71-b570-4e12-8397-eef41002e44f None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:07.784503 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73db71-b570-4e12-8397-eef41002e44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:07.882410 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-3b73db71-b570-4e12-8397-eef41002e44f None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:14:07.885828 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-3b73db71-b570-4e12-8397-eef41002e44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:07.985889 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 702/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 12:14:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:14:08.139936 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41153563-3e62-4369-8814-d61189e32694 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:14:08.141491 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-41153563-3e62-4369-8814-d61189e32694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:08.228224 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:08.291212 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41153563-3e62-4369-8814-d61189e32694 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:14:08.406500 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041581f5-c79b-4090-8bb9-b9115a27ac1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:14:08.407116 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041581f5-c79b-4090-8bb9-b9115a27ac1a service nova] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:08.407393 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041581f5-c79b-4090-8bb9-b9115a27ac1a service nova] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:08.407701 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041581f5-c79b-4090-8bb9-b9115a27ac1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:08.409338 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-041581f5-c79b-4090-8bb9-b9115a27ac1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:08.457994 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:08.501352 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041581f5-c79b-4090-8bb9-b9115a27ac1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:14:08.502974 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041581f5-c79b-4090-8bb9-b9115a27ac1a service nova] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:14:08.521706 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 726/1428] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:14:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:14:09.417729 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a32b77-adf2-4e6b-b094-0b4f79c598f8 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:09.418289 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a32b77-adf2-4e6b-b094-0b4f79c598f8 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:09.418289 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a32b77-adf2-4e6b-b094-0b4f79c598f8 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:09.420319 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 703/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 12:14:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:14:09.425098 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90d882-901e-478f-95da-e16e5fc0cf94 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:09.425501 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90d882-901e-478f-95da-e16e5fc0cf94 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:09.425666 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90d882-901e-478f-95da-e16e5fc0cf94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:09.505195 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-bf90d882-901e-478f-95da-e16e5fc0cf94 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:14:09.506423 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-bf90d882-901e-478f-95da-e16e5fc0cf94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:09.602532 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 727/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 12:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:14:09.753037 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a32b77-adf2-4e6b-b094-0b4f79c598f8 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:14:09.754607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-d0a32b77-adf2-4e6b-b094-0b4f79c598f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:09.803512 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:09.900607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a32b77-adf2-4e6b-b094-0b4f79c598f8 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:14:10.012608 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609550b5-01f8-4826-a68d-ae7fc8fc54c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:14:10.013735 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609550b5-01f8-4826-a68d-ae7fc8fc54c2 service nova] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:10.014017 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609550b5-01f8-4826-a68d-ae7fc8fc54c2 service nova] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:10.014291 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609550b5-01f8-4826-a68d-ae7fc8fc54c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:10.015934 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-609550b5-01f8-4826-a68d-ae7fc8fc54c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:10.022820 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:10.105472 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609550b5-01f8-4826-a68d-ae7fc8fc54c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:14:10.107100 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609550b5-01f8-4826-a68d-ae7fc8fc54c2 service nova] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:14:10.125695 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 704/1431] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:14:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:14:11.014752 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e27ed0-0518-4b05-a5da-5e0c26ddd7c1 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:11.015330 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e27ed0-0518-4b05-a5da-5e0c26ddd7c1 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:11.015330 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e27ed0-0518-4b05-a5da-5e0c26ddd7c1 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:11.016689 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 728/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 12:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:14:11.021784 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7dc751-b707-4b68-a8a3-9069426390a5 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:11.022231 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7dc751-b707-4b68-a8a3-9069426390a5 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:11.022403 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7dc751-b707-4b68-a8a3-9069426390a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:11.100101 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-dd7dc751-b707-4b68-a8a3-9069426390a5 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:14:11.101337 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-dd7dc751-b707-4b68-a8a3-9069426390a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:11.170416 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:11.200781 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 705/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 12:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:14:11.351838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e27ed0-0518-4b05-a5da-5e0c26ddd7c1 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:14:11.353466 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-63e27ed0-0518-4b05-a5da-5e0c26ddd7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:11.370997 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:11.502341 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e27ed0-0518-4b05-a5da-5e0c26ddd7c1 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:14:11.589327 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:11.616639 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8493cf-8e4f-4e6f-8add-5842bcbc6429 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:14:11.617247 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8493cf-8e4f-4e6f-8add-5842bcbc6429 service nova] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:11.617528 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8493cf-8e4f-4e6f-8add-5842bcbc6429 service nova] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:11.617805 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8493cf-8e4f-4e6f-8add-5842bcbc6429 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:11.619720 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-9a8493cf-8e4f-4e6f-8add-5842bcbc6429 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:11.712814 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8493cf-8e4f-4e6f-8add-5842bcbc6429 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:14:11.714437 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8493cf-8e4f-4e6f-8add-5842bcbc6429 service nova] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:14:11.732698 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 729/1434] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:14:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:14:12.618554 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0242b2-bd4c-4c2f-8b7e-0556675b78c0 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:12.619156 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0242b2-bd4c-4c2f-8b7e-0556675b78c0 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:12.619156 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0242b2-bd4c-4c2f-8b7e-0556675b78c0 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:12.620467 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 706/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 12:14:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:14:12.625943 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948badce-292d-4937-a495-87d5db6213c9 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:12.626384 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948badce-292d-4937-a495-87d5db6213c9 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:12.626491 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948badce-292d-4937-a495-87d5db6213c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:12.704543 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-948badce-292d-4937-a495-87d5db6213c9 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:14:12.705883 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-948badce-292d-4937-a495-87d5db6213c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:12.748508 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:12.803234 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 730/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 12:14:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:14:12.967066 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0242b2-bd4c-4c2f-8b7e-0556675b78c0 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:14:12.968669 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-ac0242b2-bd4c-4c2f-8b7e-0556675b78c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:13.114073 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0242b2-bd4c-4c2f-8b7e-0556675b78c0 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:14:13.169523 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:13.228417 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976898be-b96d-49e5-9937-1b6d4da7cd02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:14:13.229028 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976898be-b96d-49e5-9937-1b6d4da7cd02 service nova] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:13.229314 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976898be-b96d-49e5-9937-1b6d4da7cd02 service nova] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:13.229594 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976898be-b96d-49e5-9937-1b6d4da7cd02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:13.231233 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-976898be-b96d-49e5-9937-1b6d4da7cd02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:13.318717 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976898be-b96d-49e5-9937-1b6d4da7cd02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:14:13.320339 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976898be-b96d-49e5-9937-1b6d4da7cd02 service nova] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:14:13.339179 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 707/1437] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:14:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:14:14.252160 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc94e0-bbb0-4dca-9829-b798ba0eb1be None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:14.252745 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc94e0-bbb0-4dca-9829-b798ba0eb1be None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:14.252745 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc94e0-bbb0-4dca-9829-b798ba0eb1be None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:14.254078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 731/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 12:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:14:14.259035 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd7bd7-2eae-41cd-8caa-7f477be2a82f None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:14.259487 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd7bd7-2eae-41cd-8caa-7f477be2a82f None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:14.259632 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd7bd7-2eae-41cd-8caa-7f477be2a82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:14.336942 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-eafd7bd7-2eae-41cd-8caa-7f477be2a82f None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:14:14.338218 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-eafd7bd7-2eae-41cd-8caa-7f477be2a82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:14.340272 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:14.434735 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 708/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 12:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:14:14.615532 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbc94e0-bbb0-4dca-9829-b798ba0eb1be None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:14:14.616996 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-1bbc94e0-bbb0-4dca-9829-b798ba0eb1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:14.765924 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbc94e0-bbb0-4dca-9829-b798ba0eb1be None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:14:14.795060 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:14.881127 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d63ac3f-0261-4ba2-b962-ab56bab50eee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:14:14.881645 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d63ac3f-0261-4ba2-b962-ab56bab50eee service nova] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:14.881926 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d63ac3f-0261-4ba2-b962-ab56bab50eee service nova] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:14.882201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d63ac3f-0261-4ba2-b962-ab56bab50eee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:14.883835 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-4d63ac3f-0261-4ba2-b962-ab56bab50eee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:14.976520 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d63ac3f-0261-4ba2-b962-ab56bab50eee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:14:14.978139 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d63ac3f-0261-4ba2-b962-ab56bab50eee service nova] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:14:14.996603 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 732/1440] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:14:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:14:16.830984 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0074350-955f-4cf6-a8bc-de5a2439da1d None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:16.831525 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0074350-955f-4cf6-a8bc-de5a2439da1d None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:16.831525 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0074350-955f-4cf6-a8bc-de5a2439da1d None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:16.832814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 709/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 12:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:14:16.838243 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffa696-a52b-429b-9c83-ec29903a1591 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:16.838686 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffa696-a52b-429b-9c83-ec29903a1591 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:16.838784 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffa696-a52b-429b-9c83-ec29903a1591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:16.880578 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:16.906325 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-d7ffa696-a52b-429b-9c83-ec29903a1591 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:14:16.907577 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-d7ffa696-a52b-429b-9c83-ec29903a1591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:17.003884 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 733/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 12:14:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:14:17.295283 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0074350-955f-4cf6-a8bc-de5a2439da1d None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:14:17.296863 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-b0074350-955f-4cf6-a8bc-de5a2439da1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:17.405890 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0074350-955f-4cf6-a8bc-de5a2439da1d None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:14:17.438877 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:17.482290 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a2240-28a3-425d-83c0-cf85c6fb5003 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:14:17.482890 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a2240-28a3-425d-83c0-cf85c6fb5003 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:17.483172 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a2240-28a3-425d-83c0-cf85c6fb5003 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:17.483474 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a2240-28a3-425d-83c0-cf85c6fb5003 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:17.485081 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-058a2240-28a3-425d-83c0-cf85c6fb5003 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:17.597426 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a2240-28a3-425d-83c0-cf85c6fb5003 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:14:17.598761 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a2240-28a3-425d-83c0-cf85c6fb5003 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:14:17.616892 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 710/1443] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:14:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:14:19.782255 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89e55d-1757-469e-9431-0e5726a3c510 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:19.782918 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89e55d-1757-469e-9431-0e5726a3c510 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:19.782918 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89e55d-1757-469e-9431-0e5726a3c510 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:19.784219 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 734/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 12:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:14:19.789882 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e852cec-93cb-498b-9b99-af606952ff87 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:19.791836 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e852cec-93cb-498b-9b99-af606952ff87 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:19.791836 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e852cec-93cb-498b-9b99-af606952ff87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:19.872454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-8e852cec-93cb-498b-9b99-af606952ff87 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:14:19.873692 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-8e852cec-93cb-498b-9b99-af606952ff87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:19.888828 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:19.978407 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 711/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 12:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:14:20.279036 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc89e55d-1757-469e-9431-0e5726a3c510 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:14:20.280630 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-cc89e55d-1757-469e-9431-0e5726a3c510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:20.392164 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc89e55d-1757-469e-9431-0e5726a3c510 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:14:20.405608 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:20.469037 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925b0ae-38c6-40b3-a487-c658b1253583 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:14:20.469639 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925b0ae-38c6-40b3-a487-c658b1253583 service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:20.469920 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925b0ae-38c6-40b3-a487-c658b1253583 service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:20.470193 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925b0ae-38c6-40b3-a487-c658b1253583 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:20.471822 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-e925b0ae-38c6-40b3-a487-c658b1253583 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:20.580068 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e925b0ae-38c6-40b3-a487-c658b1253583 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:14:20.581412 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e925b0ae-38c6-40b3-a487-c658b1253583 service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:14:20.599542 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 735/1446] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:14:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:14:22.905141 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5680e4-3a2b-4668-b57f-b5dda9a8bff4 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:22.905681 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5680e4-3a2b-4668-b57f-b5dda9a8bff4 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:22.905805 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5680e4-3a2b-4668-b57f-b5dda9a8bff4 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:22.907390 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 712/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 12:14:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:14:22.912484 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a867a8c-28d2-40cc-80ad-76930c3a6a18 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:22.912906 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a867a8c-28d2-40cc-80ad-76930c3a6a18 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:22.913062 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a867a8c-28d2-40cc-80ad-76930c3a6a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:22.960559 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:22.978157 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-7a867a8c-28d2-40cc-80ad-76930c3a6a18 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:14:22.982671 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-7a867a8c-28d2-40cc-80ad-76930c3a6a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:23.075339 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 736/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 12:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:14:23.364771 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5680e4-3a2b-4668-b57f-b5dda9a8bff4 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:14:23.366319 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-2a5680e4-3a2b-4668-b57f-b5dda9a8bff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:23.475204 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5680e4-3a2b-4668-b57f-b5dda9a8bff4 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:14:23.515042 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:23.549364 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047dcf3d-e20a-454a-ace9-c6f9d4c19501 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:14:23.549975 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dcf3d-e20a-454a-ace9-c6f9d4c19501 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:23.550262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dcf3d-e20a-454a-ace9-c6f9d4c19501 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:23.550539 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dcf3d-e20a-454a-ace9-c6f9d4c19501 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:23.552182 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-047dcf3d-e20a-454a-ace9-c6f9d4c19501 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:23.663042 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047dcf3d-e20a-454a-ace9-c6f9d4c19501 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:14:23.664440 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047dcf3d-e20a-454a-ace9-c6f9d4c19501 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:14:23.683001 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 713/1449] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:14:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:14:25.705472 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9381892-3188-49e5-97fe-22fb33856835 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:25.705921 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9381892-3188-49e5-97fe-22fb33856835 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:25.706046 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9381892-3188-49e5-97fe-22fb33856835 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:25.707406 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 737/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 12:14:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:14:25.712792 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a69ab-99b5-4adc-b4cb-4d2a1be2f8dc None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:25.713240 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a69ab-99b5-4adc-b4cb-4d2a1be2f8dc None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:25.713338 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a69ab-99b5-4adc-b4cb-4d2a1be2f8dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:25.780228 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-164a69ab-99b5-4adc-b4cb-4d2a1be2f8dc None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:14:25.781452 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-164a69ab-99b5-4adc-b4cb-4d2a1be2f8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:25.802513 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:25.880475 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 714/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 12:14:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:14:26.165317 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9381892-3188-49e5-97fe-22fb33856835 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:14:26.166931 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-c9381892-3188-49e5-97fe-22fb33856835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:26.269726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9381892-3188-49e5-97fe-22fb33856835 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:14:26.290007 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:26.344268 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161286ab-799e-4a27-91d0-4ce4c8ffd509 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:14:26.344871 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161286ab-799e-4a27-91d0-4ce4c8ffd509 service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:26.345153 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161286ab-799e-4a27-91d0-4ce4c8ffd509 service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:26.345427 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161286ab-799e-4a27-91d0-4ce4c8ffd509 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:26.347024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-161286ab-799e-4a27-91d0-4ce4c8ffd509 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:26.455109 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161286ab-799e-4a27-91d0-4ce4c8ffd509 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:14:26.456580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161286ab-799e-4a27-91d0-4ce4c8ffd509 service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:14:26.474777 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 738/1452] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:14:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:14:28.518915 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0a8f2-608d-40a0-9a94-a083be6bbb67 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:28.519505 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0a8f2-608d-40a0-9a94-a083be6bbb67 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:28.519505 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0a8f2-608d-40a0-9a94-a083be6bbb67 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:28.520895 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 715/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 12:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:14:28.526111 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84387320-9963-405b-bba6-70d0f0ed8774 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:28.526552 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84387320-9963-405b-bba6-70d0f0ed8774 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:28.526712 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84387320-9963-405b-bba6-70d0f0ed8774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:28.592033 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:28.594798 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-84387320-9963-405b-bba6-70d0f0ed8774 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:14:28.596092 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-84387320-9963-405b-bba6-70d0f0ed8774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:28.694209 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 739/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 12:14:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:14:28.977299 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff0a8f2-608d-40a0-9a94-a083be6bbb67 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:14:28.978855 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-7ff0a8f2-608d-40a0-9a94-a083be6bbb67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:29.081761 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff0a8f2-608d-40a0-9a94-a083be6bbb67 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:14:29.132020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:29.157525 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f362f7f0-1c65-43d7-82f9-1177ea06d2bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:14:29.158134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362f7f0-1c65-43d7-82f9-1177ea06d2bc service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:29.158442 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362f7f0-1c65-43d7-82f9-1177ea06d2bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:29.158719 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362f7f0-1c65-43d7-82f9-1177ea06d2bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:29.160396 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-f362f7f0-1c65-43d7-82f9-1177ea06d2bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:29.269648 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f362f7f0-1c65-43d7-82f9-1177ea06d2bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:14:29.271089 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f362f7f0-1c65-43d7-82f9-1177ea06d2bc service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:14:29.289263 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 716/1455] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:14:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:14:31.263188 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde04a0a-7512-45e0-9e54-4ccc7de9b3f7 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:31.263771 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde04a0a-7512-45e0-9e54-4ccc7de9b3f7 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:31.263771 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde04a0a-7512-45e0-9e54-4ccc7de9b3f7 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:31.265138 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 740/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 12:14:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:14:31.270830 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79e99d-0f23-43ae-8bbe-af4235b7bb81 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:31.271304 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79e99d-0f23-43ae-8bbe-af4235b7bb81 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:31.271440 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79e99d-0f23-43ae-8bbe-af4235b7bb81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:31.350694 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-fe79e99d-0f23-43ae-8bbe-af4235b7bb81 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:14:31.352004 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-fe79e99d-0f23-43ae-8bbe-af4235b7bb81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:31.381442 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:31.452982 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 717/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 12:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:14:31.748238 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde04a0a-7512-45e0-9e54-4ccc7de9b3f7 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:14:31.749843 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-cde04a0a-7512-45e0-9e54-4ccc7de9b3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:31.853387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde04a0a-7512-45e0-9e54-4ccc7de9b3f7 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:14:31.881509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:31.925992 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e9b4d-2f88-4bde-a55c-302bc1397370 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:14:31.926618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e9b4d-2f88-4bde-a55c-302bc1397370 service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:31.926902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e9b4d-2f88-4bde-a55c-302bc1397370 service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:31.927207 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e9b4d-2f88-4bde-a55c-302bc1397370 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:31.928813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-186e9b4d-2f88-4bde-a55c-302bc1397370 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:32.038428 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e9b4d-2f88-4bde-a55c-302bc1397370 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:14:32.039896 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e9b4d-2f88-4bde-a55c-302bc1397370 service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:14:32.058330 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 741/1458] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:14:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:14:34.056171 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afae5d-affe-4414-b45f-b1afd44c97a6 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:34.056769 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afae5d-affe-4414-b45f-b1afd44c97a6 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:34.056769 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afae5d-affe-4414-b45f-b1afd44c97a6 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:34.058633 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 718/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 12:14:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:14:34.063913 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b3d96-3857-41cf-aba6-3ae6b5414cca None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:34.064339 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b3d96-3857-41cf-aba6-3ae6b5414cca None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:34.064479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b3d96-3857-41cf-aba6-3ae6b5414cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:34.131766 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-501b3d96-3857-41cf-aba6-3ae6b5414cca None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:14:34.132988 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-501b3d96-3857-41cf-aba6-3ae6b5414cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:34.141926 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:34.231155 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 742/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 12:14:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:14:34.516499 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35afae5d-affe-4414-b45f-b1afd44c97a6 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:14:34.518049 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-35afae5d-affe-4414-b45f-b1afd44c97a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:34.622182 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35afae5d-affe-4414-b45f-b1afd44c97a6 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:14:34.677546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:34.695873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc8bee-29f4-49cd-b0ac-319dbce8e5ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:14:34.707188 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc8bee-29f4-49cd-b0ac-319dbce8e5ae service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:34.707188 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc8bee-29f4-49cd-b0ac-319dbce8e5ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:34.707188 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc8bee-29f4-49cd-b0ac-319dbce8e5ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:34.707188 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-1dfc8bee-29f4-49cd-b0ac-319dbce8e5ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:34.807384 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfc8bee-29f4-49cd-b0ac-319dbce8e5ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:14:34.808781 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfc8bee-29f4-49cd-b0ac-319dbce8e5ae service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:14:34.827960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 719/1461] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:14:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:14:36.788901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5215b-92f5-434a-883a-b1162899d5c4 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:36.789522 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5215b-92f5-434a-883a-b1162899d5c4 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:36.789522 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5215b-92f5-434a-883a-b1162899d5c4 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:36.790808 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 743/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 12:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:14:36.796212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f237120-f933-479d-9e0c-c89900c44aca None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:36.796603 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f237120-f933-479d-9e0c-c89900c44aca None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:36.796773 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f237120-f933-479d-9e0c-c89900c44aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:36.863050 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-5f237120-f933-479d-9e0c-c89900c44aca None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:14:36.864247 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-5f237120-f933-479d-9e0c-c89900c44aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:36.903578 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:36.960741 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 720/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 12:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:14:37.249448 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5215b-92f5-434a-883a-b1162899d5c4 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:14:37.251038 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-cdd5215b-92f5-434a-883a-b1162899d5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:37.358944 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5215b-92f5-434a-883a-b1162899d5c4 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:14:37.401033 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:37.437410 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cc8a37-bd0d-4f00-96eb-5ea688aa1a57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:14:37.437930 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc8a37-bd0d-4f00-96eb-5ea688aa1a57 service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:37.438214 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc8a37-bd0d-4f00-96eb-5ea688aa1a57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:37.438489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc8a37-bd0d-4f00-96eb-5ea688aa1a57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:37.440182 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-47cc8a37-bd0d-4f00-96eb-5ea688aa1a57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:37.548979 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cc8a37-bd0d-4f00-96eb-5ea688aa1a57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:14:37.550327 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cc8a37-bd0d-4f00-96eb-5ea688aa1a57 service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:14:37.568800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 744/1464] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:14:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:14:39.534014 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33aa00d-ba7f-4d7b-8dcd-cad821926d50 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:39.534619 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33aa00d-ba7f-4d7b-8dcd-cad821926d50 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:39.534619 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33aa00d-ba7f-4d7b-8dcd-cad821926d50 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:39.535984 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 721/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 12:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:14:39.542497 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bd853-311d-46d9-bf8f-d2afbde43270 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:39.542933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bd853-311d-46d9-bf8f-d2afbde43270 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:39.543043 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bd853-311d-46d9-bf8f-d2afbde43270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:39.609441 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-d35bd853-311d-46d9-bf8f-d2afbde43270 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:14:39.610696 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-d35bd853-311d-46d9-bf8f-d2afbde43270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:39.628928 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:39.709693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 745/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 12:14:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:14:39.990888 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33aa00d-ba7f-4d7b-8dcd-cad821926d50 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:14:39.992520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-b33aa00d-ba7f-4d7b-8dcd-cad821926d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:40.101989 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33aa00d-ba7f-4d7b-8dcd-cad821926d50 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:14:40.171212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:40.177988 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66594f1-e6bd-4b50-89cd-5d248c382a47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:14:40.178610 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66594f1-e6bd-4b50-89cd-5d248c382a47 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:40.178882 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66594f1-e6bd-4b50-89cd-5d248c382a47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:40.179159 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66594f1-e6bd-4b50-89cd-5d248c382a47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:40.180842 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-f66594f1-e6bd-4b50-89cd-5d248c382a47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:40.289805 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66594f1-e6bd-4b50-89cd-5d248c382a47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:14:40.291777 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66594f1-e6bd-4b50-89cd-5d248c382a47 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:14:40.309735 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 722/1467] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:14:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:14:42.235694 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee473872-6d2b-46b3-9b1e-4e11a95cd518 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:42.236300 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee473872-6d2b-46b3-9b1e-4e11a95cd518 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:42.236300 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee473872-6d2b-46b3-9b1e-4e11a95cd518 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:42.237602 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 746/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 12:14:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:14:42.242857 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40e6f4-bade-47b1-9a73-d5dbe2af56f0 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:42.243367 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40e6f4-bade-47b1-9a73-d5dbe2af56f0 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:42.243487 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40e6f4-bade-47b1-9a73-d5dbe2af56f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:42.309905 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-fc40e6f4-bade-47b1-9a73-d5dbe2af56f0 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:14:42.311118 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-fc40e6f4-bade-47b1-9a73-d5dbe2af56f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:42.359683 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:42.414774 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 723/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 12:14:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:14:42.699195 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee473872-6d2b-46b3-9b1e-4e11a95cd518 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:14:42.700822 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-ee473872-6d2b-46b3-9b1e-4e11a95cd518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:42.803053 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee473872-6d2b-46b3-9b1e-4e11a95cd518 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:14:42.850492 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:42.876798 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf72e1-735c-42bb-ac5c-7374ad344dee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:14:42.877425 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf72e1-735c-42bb-ac5c-7374ad344dee service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:14:42.877712 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf72e1-735c-42bb-ac5c-7374ad344dee service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:14:42.877991 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf72e1-735c-42bb-ac5c-7374ad344dee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:14:42.880436 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-36cf72e1-735c-42bb-ac5c-7374ad344dee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:14:42.988473 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cf72e1-735c-42bb-ac5c-7374ad344dee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:14:42.989920 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cf72e1-735c-42bb-ac5c-7374ad344dee service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:14:43.008117 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 747/1470] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:14:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:14:52.412959 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=22 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:14:53.005823 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=66 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:15:02.841753 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdae971-6efb-4af5-9425-a39ad24395c1 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:15:02.842317 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdae971-6efb-4af5-9425-a39ad24395c1 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:15:02.842317 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdae971-6efb-4af5-9425-a39ad24395c1 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:15:02.843726 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 724/1471] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:15:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:15:02.848850 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e627d96-ebcd-4520-9a01-b36056a79e2b None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:15:02.849305 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e627d96-ebcd-4520-9a01-b36056a79e2b None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:15:02.849435 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e627d96-ebcd-4520-9a01-b36056a79e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:15:02.915319 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-8e627d96-ebcd-4520-9a01-b36056a79e2b None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:15:02.916565 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-8e627d96-ebcd-4520-9a01-b36056a79e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:15:03.013398 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 748/1472] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:15:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:15:03.318424 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc488b-005d-4a71-903a-07b650a02fc8 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:15:03.318720 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc488b-005d-4a71-903a-07b650a02fc8 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:15:03.319002 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc488b-005d-4a71-903a-07b650a02fc8 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:15:03.320850 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 725/1473] 10.222.0.98 () {54 vars in 967 bytes} [Thu Aug 18 12:15:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:15:03.325781 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eebf9d8-6656-4d86-ab1a-cf83c2ba61f8 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:15:03.326072 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eebf9d8-6656-4d86-ab1a-cf83c2ba61f8 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:15:03.326350 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eebf9d8-6656-4d86-ab1a-cf83c2ba61f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:15:03.402923 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-5eebf9d8-6656-4d86-ab1a-cf83c2ba61f8 None None] MFA Rules not processed for user `d020063702b0481e9f4111680766f7fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:15:03.404265 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-5eebf9d8-6656-4d86-ab1a-cf83c2ba61f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:15:03.424743 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:15:03.480111 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 749/1474] 10.222.0.98 () {58 vars in 1064 bytes} [Thu Aug 18 12:15:03 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:15:03.487649 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc488b-005d-4a71-903a-07b650a02fc8 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:15:03.489190 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-f3fc488b-005d-4a71-903a-07b650a02fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:15:03.594148 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc488b-005d-4a71-903a-07b650a02fc8 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:15:03.667448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b2046e-0bbc-4da9-b0e8-a0d6d823746f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd020063702b0481e9f4111680766f7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:15:03.668065 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2046e-0bbc-4da9-b0e8-a0d6d823746f service glance] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:15:03.668345 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2046e-0bbc-4da9-b0e8-a0d6d823746f service glance] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:15:03.668622 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2046e-0bbc-4da9-b0e8-a0d6d823746f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:15:03.670413 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-65b2046e-0bbc-4da9-b0e8-a0d6d823746f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:15:03.731540 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:15:03.794069 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b2046e-0bbc-4da9-b0e8-a0d6d823746f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:15:03.795548 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b2046e-0bbc-4da9-b0e8-a0d6d823746f service glance] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:15:03.814769 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 726/1475] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:15:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:15:05.801241 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a5863-6444-4b45-ba69-72a20cd0d16a None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:15:05.801759 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a5863-6444-4b45-ba69-72a20cd0d16a None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:15:05.801759 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a5863-6444-4b45-ba69-72a20cd0d16a None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:15:05.803175 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 750/1476] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:15:05.808925 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d72a60-5e95-4b46-b4da-73c4b066be6d None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:15:05.809349 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d72a60-5e95-4b46-b4da-73c4b066be6d None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:15:05.809484 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d72a60-5e95-4b46-b4da-73c4b066be6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:15:05.876241 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-53d72a60-5e95-4b46-b4da-73c4b066be6d None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:15:05.877463 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-53d72a60-5e95-4b46-b4da-73c4b066be6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:15:05.973201 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 727/1477] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:15:06.254162 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9a5863-6444-4b45-ba69-72a20cd0d16a None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:15:06.255740 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-4f9a5863-6444-4b45-ba69-72a20cd0d16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:15:06.381393 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9a5863-6444-4b45-ba69-72a20cd0d16a None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:15:06.430722 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:15:06.476524 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbcde21-f5be-49dc-a4f2-d9eff5218e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:15:06.477136 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbcde21-f5be-49dc-a4f2-d9eff5218e1a admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:15:06.477425 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbcde21-f5be-49dc-a4f2-d9eff5218e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:15:06.477706 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbcde21-f5be-49dc-a4f2-d9eff5218e1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:15:06.478346 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbcde21-f5be-49dc-a4f2-d9eff5218e1a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:15:06.480171 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: WARNING py.warnings [None req-5cbcde21-f5be-49dc-a4f2-d9eff5218e1a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 12:15:06.480171 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: warnings.warn(msg) Aug 18 12:15:06.480171 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: Aug 18 12:15:06.481322 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbcde21-f5be-49dc-a4f2-d9eff5218e1a admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:15:06.500789 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 751/1478] 10.222.0.98 () {56 vars in 1108 bytes} [Thu Aug 18 12:15:06 2022] GET /identity/v3/projects => generated 2553 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:15:08.450754 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55167c27-3d37-4f11-af89-cd4508bc916f None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:15:08.451330 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55167c27-3d37-4f11-af89-cd4508bc916f None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:15:08.451330 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55167c27-3d37-4f11-af89-cd4508bc916f None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:15:08.452690 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 728/1479] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:15:08.457929 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f732c9-b797-49bc-8fb1-96ecc73fb433 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:15:08.458362 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f732c9-b797-49bc-8fb1-96ecc73fb433 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:15:08.458494 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f732c9-b797-49bc-8fb1-96ecc73fb433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:15:08.545223 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-65f732c9-b797-49bc-8fb1-96ecc73fb433 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:15:08.546813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-65f732c9-b797-49bc-8fb1-96ecc73fb433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:15:08.647970 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 752/1480] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:15:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:15:08.950264 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55167c27-3d37-4f11-af89-cd4508bc916f None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:15:08.951877 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-55167c27-3d37-4f11-af89-cd4508bc916f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:15:08.962246 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:15:09.099395 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55167c27-3d37-4f11-af89-cd4508bc916f None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:15:09.182765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:15:09.219168 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ecc223-b9cb-4c11-a418-42aa757f2c46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:15:09.219809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ecc223-b9cb-4c11-a418-42aa757f2c46 service nova] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:15:09.220095 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ecc223-b9cb-4c11-a418-42aa757f2c46 service nova] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:15:09.220374 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ecc223-b9cb-4c11-a418-42aa757f2c46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:15:09.222173 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-f3ecc223-b9cb-4c11-a418-42aa757f2c46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:15:09.312288 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ecc223-b9cb-4c11-a418-42aa757f2c46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:15:09.313881 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ecc223-b9cb-4c11-a418-42aa757f2c46 service nova] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:15:09.331655 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 729/1481] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:15:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:15:11.448804 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e9249-47b8-47a2-a5d5-248061e0017f None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:15:11.449365 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e9249-47b8-47a2-a5d5-248061e0017f None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:15:11.449365 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e9249-47b8-47a2-a5d5-248061e0017f None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:15:11.450722 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 753/1482] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:15:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:15:11.456110 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e21b33-f510-4521-bb97-10db69a6132d None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:15:11.456529 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e21b33-f510-4521-bb97-10db69a6132d None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:15:11.456792 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e21b33-f510-4521-bb97-10db69a6132d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:15:11.522756 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-e2e21b33-f510-4521-bb97-10db69a6132d None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:15:11.524008 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-e2e21b33-f510-4521-bb97-10db69a6132d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:15:11.571034 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:15:11.621247 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 730/1483] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:15:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:15:11.906901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e9249-47b8-47a2-a5d5-248061e0017f None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:15:11.908503 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-8d7e9249-47b8-47a2-a5d5-248061e0017f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:15:11.932638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:15:12.013431 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e9249-47b8-47a2-a5d5-248061e0017f None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:15:12.087199 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c15d0-cfb7-4d7e-bd3f-2f39ef806481 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:15:12.087852 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c15d0-cfb7-4d7e-bd3f-2f39ef806481 service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:15:12.088144 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c15d0-cfb7-4d7e-bd3f-2f39ef806481 service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:15:12.088419 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c15d0-cfb7-4d7e-bd3f-2f39ef806481 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:15:12.090280 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-763c15d0-cfb7-4d7e-bd3f-2f39ef806481 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:15:12.166656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:15:12.201949 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763c15d0-cfb7-4d7e-bd3f-2f39ef806481 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:15:12.203389 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763c15d0-cfb7-4d7e-bd3f-2f39ef806481 service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:15:12.222303 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 754/1484] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:15:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:15:14.186537 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b9054-f98e-49a2-802f-b7eee4271d72 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:15:14.187145 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b9054-f98e-49a2-802f-b7eee4271d72 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:15:14.187145 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b9054-f98e-49a2-802f-b7eee4271d72 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:15:14.188526 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 731/1485] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:15:14.193821 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad70f8e9-521e-43cf-9466-930b1636538a None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:15:14.194267 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad70f8e9-521e-43cf-9466-930b1636538a None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:15:14.194425 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad70f8e9-521e-43cf-9466-930b1636538a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:15:14.261105 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-ad70f8e9-521e-43cf-9466-930b1636538a None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:15:14.262369 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-ad70f8e9-521e-43cf-9466-930b1636538a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:15:14.358336 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 755/1486] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:15:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:15:14.645023 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b9054-f98e-49a2-802f-b7eee4271d72 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:15:14.647191 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-366b9054-f98e-49a2-802f-b7eee4271d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:15:14.755755 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b9054-f98e-49a2-802f-b7eee4271d72 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:15:14.828981 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:15:14.831985 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b81926-6344-4112-a5af-81063c710e3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:15:14.832982 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b81926-6344-4112-a5af-81063c710e3b service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:15:14.833459 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b81926-6344-4112-a5af-81063c710e3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:15:14.833954 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b81926-6344-4112-a5af-81063c710e3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:15:14.836756 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-44b81926-6344-4112-a5af-81063c710e3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:15:14.950228 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b81926-6344-4112-a5af-81063c710e3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:15:14.951709 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b81926-6344-4112-a5af-81063c710e3b service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:15:15.087653 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 732/1487] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:15:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:15:17.161545 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e3ca07-ded1-4cc1-b11a-30b1591f4366 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:15:17.162667 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e3ca07-ded1-4cc1-b11a-30b1591f4366 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:15:17.162667 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e3ca07-ded1-4cc1-b11a-30b1591f4366 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:15:17.164049 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 756/1488] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:15:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:15:17.169369 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c1d04-06d5-4d82-b3cd-86e52fbccc49 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:15:17.169829 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c1d04-06d5-4d82-b3cd-86e52fbccc49 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:15:17.169939 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c1d04-06d5-4d82-b3cd-86e52fbccc49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:15:17.242984 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-ef4c1d04-06d5-4d82-b3cd-86e52fbccc49 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:15:17.244244 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-ef4c1d04-06d5-4d82-b3cd-86e52fbccc49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:15:17.297342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:15:17.340976 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 733/1489] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:15:17.643086 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e3ca07-ded1-4cc1-b11a-30b1591f4366 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:15:17.644707 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-63e3ca07-ded1-4cc1-b11a-30b1591f4366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:15:17.682736 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:15:17.876149 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e3ca07-ded1-4cc1-b11a-30b1591f4366 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:15:17.965101 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b717f219-ac28-4df4-927f-5f565b8d1400 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:15:17.965712 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717f219-ac28-4df4-927f-5f565b8d1400 service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:15:17.965999 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717f219-ac28-4df4-927f-5f565b8d1400 service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:15:17.966279 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717f219-ac28-4df4-927f-5f565b8d1400 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:15:17.967961 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-b717f219-ac28-4df4-927f-5f565b8d1400 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:15:18.049100 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:15:18.078543 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b717f219-ac28-4df4-927f-5f565b8d1400 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:15:18.080047 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b717f219-ac28-4df4-927f-5f565b8d1400 service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:15:18.098683 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 757/1490] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:15:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:15:20.464110 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc86781-5684-4fb5-90e2-9f0af21de069 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:15:20.464773 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc86781-5684-4fb5-90e2-9f0af21de069 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:15:20.464773 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc86781-5684-4fb5-90e2-9f0af21de069 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:15:20.465978 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 734/1491] 10.222.0.98 () {54 vars in 927 bytes} [Thu Aug 18 12:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:15:20.471324 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca5135-370e-432b-92ac-6cd0e8ba20ed None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:15:20.471815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca5135-370e-432b-92ac-6cd0e8ba20ed None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:15:20.471929 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca5135-370e-432b-92ac-6cd0e8ba20ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:15:20.537165 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-56ca5135-370e-432b-92ac-6cd0e8ba20ed None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:15:20.538399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-56ca5135-370e-432b-92ac-6cd0e8ba20ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:15:20.635086 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 758/1492] 10.222.0.98 () {58 vars in 1024 bytes} [Thu Aug 18 12:15:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:15:20.925596 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc86781-5684-4fb5-90e2-9f0af21de069 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:15:20.927182 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-ebc86781-5684-4fb5-90e2-9f0af21de069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:15:21.030571 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc86781-5684-4fb5-90e2-9f0af21de069 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:15:21.105038 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d3357f-e778-4f44-a918-b7a08b629a47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:15:21.105639 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d3357f-e778-4f44-a918-b7a08b629a47 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:15:21.105920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d3357f-e778-4f44-a918-b7a08b629a47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:15:21.106195 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d3357f-e778-4f44-a918-b7a08b629a47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:15:21.107835 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-93d3357f-e778-4f44-a918-b7a08b629a47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:15:21.117215 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:15:21.218541 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d3357f-e778-4f44-a918-b7a08b629a47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:15:21.220016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d3357f-e778-4f44-a918-b7a08b629a47 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:15:21.238770 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 735/1493] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:15:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:15:30.633829 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=82 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:15:31.236514 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=50 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:16:49.991652 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb493a9-e300-4e22-b3c7-b68ad762664a None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:16:49.992245 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb493a9-e300-4e22-b3c7-b68ad762664a None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:16:49.992245 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb493a9-e300-4e22-b3c7-b68ad762664a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:16:50.068568 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-3bb493a9-e300-4e22-b3c7-b68ad762664a None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:16:50.069744 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-3bb493a9-e300-4e22-b3c7-b68ad762664a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:16:50.166031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 759/1494] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:16:50.174336 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cb508-c991-405e-bb03-88b8d1009656 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:16:50.174864 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cb508-c991-405e-bb03-88b8d1009656 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:16:50.174951 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cb508-c991-405e-bb03-88b8d1009656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:16:50.256768 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-1e7cb508-c991-405e-bb03-88b8d1009656 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:16:50.257930 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-1e7cb508-c991-405e-bb03-88b8d1009656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:16:50.358978 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 736/1495] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:16:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:16:50.363693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb493a9-e300-4e22-b3c7-b68ad762664a None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:16:50.365525 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-3bb493a9-e300-4e22-b3c7-b68ad762664a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:16:50.377546 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb493a9-e300-4e22-b3c7-b68ad762664a None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:16:50.439936 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:16:50.472800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b29658e-a3ad-4efe-a449-877b3be79578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:16:50.473941 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b29658e-a3ad-4efe-a449-877b3be79578 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:16:50.474224 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b29658e-a3ad-4efe-a449-877b3be79578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:16:50.474496 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b29658e-a3ad-4efe-a449-877b3be79578 admin admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:16:50.475139 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b29658e-a3ad-4efe-a449-877b3be79578 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:16:50.475463 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b29658e-a3ad-4efe-a449-877b3be79578 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:16:50.476480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: WARNING py.warnings [None req-8b29658e-a3ad-4efe-a449-877b3be79578 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 18 12:16:50.476480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: warnings.warn(msg) Aug 18 12:16:50.476480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: Aug 18 12:16:50.477371 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b29658e-a3ad-4efe-a449-877b3be79578 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:16:50.488555 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 760/1496] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:16:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:16:50.497453 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7cb508-c991-405e-bb03-88b8d1009656 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:16:50.507101 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7cb508-c991-405e-bb03-88b8d1009656 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:16:50.570181 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:16:50.606148 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184796d3-bc4d-4a42-8e89-aa776add3742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:16:50.606780 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184796d3-bc4d-4a42-8e89-aa776add3742 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:16:50.607063 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184796d3-bc4d-4a42-8e89-aa776add3742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:16:50.607338 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184796d3-bc4d-4a42-8e89-aa776add3742 admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:16:50.607998 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184796d3-bc4d-4a42-8e89-aa776add3742 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:16:50.609774 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: WARNING py.warnings [None req-184796d3-bc4d-4a42-8e89-aa776add3742 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 12:16:50.609774 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: warnings.warn(msg) Aug 18 12:16:50.609774 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: Aug 18 12:16:50.610984 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184796d3-bc4d-4a42-8e89-aa776add3742 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:16:50.739826 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 737/1497] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:16:50 2022] POST /identity/v3/projects => generated 377 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:16:50.744466 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b29658e-a3ad-4efe-a449-877b3be79578 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:16:50.746013 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-8b29658e-a3ad-4efe-a449-877b3be79578 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:16:50.780867 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b29658e-a3ad-4efe-a449-877b3be79578 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:16:50.874231 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70e1d9-00a1-4fed-bf91-73c47de00037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:16:50.874840 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70e1d9-00a1-4fed-bf91-73c47de00037 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:16:50.875127 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70e1d9-00a1-4fed-bf91-73c47de00037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:16:50.875406 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70e1d9-00a1-4fed-bf91-73c47de00037 admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:16:50.876070 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70e1d9-00a1-4fed-bf91-73c47de00037 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:16:50.877094 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: WARNING py.warnings [None req-7c70e1d9-00a1-4fed-bf91-73c47de00037 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 12:16:50.877094 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: warnings.warn(msg) Aug 18 12:16:50.877094 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: Aug 18 12:16:50.878563 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70e1d9-00a1-4fed-bf91-73c47de00037 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:16:50.998539 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 761/1498] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:16:50 2022] POST /identity/v3/users => generated 289 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:16:51.003013 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184796d3-bc4d-4a42-8e89-aa776add3742 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:16:51.005123 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-184796d3-bc4d-4a42-8e89-aa776add3742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:16:51.066616 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184796d3-bc4d-4a42-8e89-aa776add3742 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:16:51.153684 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:16:51.164994 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f53f8-a57e-4756-8f3e-dc0f99adfe5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:16:51.167454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f53f8-a57e-4756-8f3e-dc0f99adfe5e admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:16:51.167454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f53f8-a57e-4756-8f3e-dc0f99adfe5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:16:51.167454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f53f8-a57e-4756-8f3e-dc0f99adfe5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:16:51.167454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558f53f8-a57e-4756-8f3e-dc0f99adfe5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:16:51.167802 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: WARNING py.warnings [None req-558f53f8-a57e-4756-8f3e-dc0f99adfe5e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 18 12:16:51.167802 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: warnings.warn(msg) Aug 18 12:16:51.167802 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: Aug 18 12:16:51.168683 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558f53f8-a57e-4756-8f3e-dc0f99adfe5e admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:16:51.181576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 738/1499] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:16:51.185724 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70e1d9-00a1-4fed-bf91-73c47de00037 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:16:51.187344 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-7c70e1d9-00a1-4fed-bf91-73c47de00037 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:16:51.199656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70e1d9-00a1-4fed-bf91-73c47de00037 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:16:51.225680 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:16:51.298226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9162b675-9381-4a0f-93ee-b923e1c5cebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:16:51.298833 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162b675-9381-4a0f-93ee-b923e1c5cebe admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:16:51.299110 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162b675-9381-4a0f-93ee-b923e1c5cebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:16:51.299375 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162b675-9381-4a0f-93ee-b923e1c5cebe admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:16:51.300031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9162b675-9381-4a0f-93ee-b923e1c5cebe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:16:51.301068 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: WARNING py.warnings [None req-9162b675-9381-4a0f-93ee-b923e1c5cebe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 18 12:16:51.301068 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: warnings.warn(msg) Aug 18 12:16:51.301068 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: Aug 18 12:16:51.302527 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9162b675-9381-4a0f-93ee-b923e1c5cebe admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:16:51.314574 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 762/1500] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:16:51.319585 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f53f8-a57e-4756-8f3e-dc0f99adfe5e admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:16:51.321115 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-558f53f8-a57e-4756-8f3e-dc0f99adfe5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:16:51.353939 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f53f8-a57e-4756-8f3e-dc0f99adfe5e admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:16:51.450943 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c428535-8597-4210-8b9e-a7e405cdbfc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:16:51.451627 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c428535-8597-4210-8b9e-a7e405cdbfc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:16:51.451912 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c428535-8597-4210-8b9e-a7e405cdbfc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:16:51.452188 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c428535-8597-4210-8b9e-a7e405cdbfc0 admin admin] PATH_INFO: `/v3/projects/31525ee21bb04fcbbec183fee166a966/users/92cab093569b41ecb21644b0201aae39/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:16:51.473453 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c428535-8597-4210-8b9e-a7e405cdbfc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31525ee21bb04fcbbec183fee166a966, user_id=92cab093569b41ecb21644b0201aae39, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:16:51.474477 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: WARNING py.warnings [None req-6c428535-8597-4210-8b9e-a7e405cdbfc0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 12:16:51.474477 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: warnings.warn(msg) Aug 18 12:16:51.474477 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: Aug 18 12:16:51.476522 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c428535-8597-4210-8b9e-a7e405cdbfc0 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:16:51.572315 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 739/1501] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:16:51 2022] PUT /identity/v3/projects/31525ee21bb04fcbbec183fee166a966/users/92cab093569b41ecb21644b0201aae39/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:16:51.584492 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6e6fe-3cc8-480a-a156-83f5dbf226d1 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:16:51.584780 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6e6fe-3cc8-480a-a156-83f5dbf226d1 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:16:51.585090 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6e6fe-3cc8-480a-a156-83f5dbf226d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:16:51.643983 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-7ed6e6fe-3cc8-480a-a156-83f5dbf226d1 None None] MFA Rules not processed for user `92cab093569b41ecb21644b0201aae39`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:16:51.645238 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-7ed6e6fe-3cc8-480a-a156-83f5dbf226d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:16:51.722374 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 763/1502] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:16:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:16:51.736303 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663cbeb-5d22-4600-953e-eedc5b5640bb None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:16:51.736596 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663cbeb-5d22-4600-953e-eedc5b5640bb None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:16:51.736880 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663cbeb-5d22-4600-953e-eedc5b5640bb None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:16:51.738217 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 740/1503] 10.222.0.98 () {54 vars in 964 bytes} [Thu Aug 18 12:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:16:51.743855 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3056f5-6e68-4f57-9574-1a6463d44226 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:16:51.744142 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3056f5-6e68-4f57-9574-1a6463d44226 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:16:51.744416 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3056f5-6e68-4f57-9574-1a6463d44226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:16:51.775242 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:16:51.821044 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-fe3056f5-6e68-4f57-9574-1a6463d44226 None None] MFA Rules not processed for user `c41b6fb3cb064a49b9236077a8faed54`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:16:51.822280 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-fe3056f5-6e68-4f57-9574-1a6463d44226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:16:51.903270 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 764/1504] 10.222.0.98 () {58 vars in 1061 bytes} [Thu Aug 18 12:16:51 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:16:51.913139 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d663cbeb-5d22-4600-953e-eedc5b5640bb None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:16:51.914714 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-d663cbeb-5d22-4600-953e-eedc5b5640bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:16:51.965564 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:16:52.020613 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d663cbeb-5d22-4600-953e-eedc5b5640bb None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:16:52.094269 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752b8790-854f-4533-a436-9d2a4b245272 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:16:52.094876 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b8790-854f-4533-a436-9d2a4b245272 service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:16:52.095157 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b8790-854f-4533-a436-9d2a4b245272 service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:16:52.095454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b8790-854f-4533-a436-9d2a4b245272 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:16:52.097071 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-752b8790-854f-4533-a436-9d2a4b245272 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:16:52.198375 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752b8790-854f-4533-a436-9d2a4b245272 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:16:52.199819 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752b8790-854f-4533-a436-9d2a4b245272 service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:16:52.203321 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:16:52.220038 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 741/1505] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:16:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:16:52.265977 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfdc17-10e6-465e-89fe-99a26c89a0a5 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:16:52.267277 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfdc17-10e6-465e-89fe-99a26c89a0a5 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:16:52.267277 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfdc17-10e6-465e-89fe-99a26c89a0a5 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:16:52.268067 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 765/1506] 10.222.0.98 () {52 vars in 830 bytes} [Thu Aug 18 12:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:16:52.272120 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752b8790-854f-4533-a436-9d2a4b245272 service cinder] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:16:52.273374 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-752b8790-854f-4533-a436-9d2a4b245272 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:16:52.374357 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752b8790-854f-4533-a436-9d2a4b245272 service cinder] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:16:52.469725 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df8a5ec-bdf2-4662-b29d-199a206cab43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:16:52.470360 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8a5ec-bdf2-4662-b29d-199a206cab43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:16:52.470641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8a5ec-bdf2-4662-b29d-199a206cab43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:16:52.470915 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8a5ec-bdf2-4662-b29d-199a206cab43 admin admin] PATH_INFO: `/v3/users/92cab093569b41ecb21644b0201aae39` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:16:52.483827 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df8a5ec-bdf2-4662-b29d-199a206cab43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92cab093569b41ecb21644b0201aae39)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:16:52.484887 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: WARNING py.warnings [None req-1df8a5ec-bdf2-4662-b29d-199a206cab43 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 12:16:52.484887 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: warnings.warn(msg) Aug 18 12:16:52.484887 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: Aug 18 12:16:52.486099 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df8a5ec-bdf2-4662-b29d-199a206cab43 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:16:52.497071 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:16:52.797068 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-1df8a5ec-bdf2-4662-b29d-199a206cab43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92cab093569b41ecb21644b0201aae39'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:16:52.797432 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-1df8a5ec-bdf2-4662-b29d-199a206cab43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92cab093569b41ecb21644b0201aae39'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:16:52.797959 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-1df8a5ec-bdf2-4662-b29d-199a206cab43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92cab093569b41ecb21644b0201aae39'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:16:52.812473 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-1df8a5ec-bdf2-4662-b29d-199a206cab43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92cab093569b41ecb21644b0201aae39'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:16:52.835084 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-1df8a5ec-bdf2-4662-b29d-199a206cab43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92cab093569b41ecb21644b0201aae39'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:16:52.928102 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 742/1507] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:16:52 2022] DELETE /identity/v3/users/92cab093569b41ecb21644b0201aae39 => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:16:52.932597 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80baf0a-2555-496a-b8b8-1b51aa9e496f None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:16:52.932888 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80baf0a-2555-496a-b8b8-1b51aa9e496f None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:16:52.933165 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80baf0a-2555-496a-b8b8-1b51aa9e496f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:16:52.990079 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-e80baf0a-2555-496a-b8b8-1b51aa9e496f None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:16:52.991307 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-e80baf0a-2555-496a-b8b8-1b51aa9e496f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:16:53.066503 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:16:53.078380 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 766/1508] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:16:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:16:53.089377 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df8a5ec-bdf2-4662-b29d-199a206cab43 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:16:53.090614 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-1df8a5ec-bdf2-4662-b29d-199a206cab43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:16:53.192137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df8a5ec-bdf2-4662-b29d-199a206cab43 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:16:53.246963 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:16:53.267471 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e075f6-f6a1-475d-b5cb-e8995bff1f09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:16:53.268146 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e075f6-f6a1-475d-b5cb-e8995bff1f09 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:16:53.268457 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e075f6-f6a1-475d-b5cb-e8995bff1f09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:16:53.268735 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e075f6-f6a1-475d-b5cb-e8995bff1f09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:16:53.270397 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-22e075f6-f6a1-475d-b5cb-e8995bff1f09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:16:53.381736 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e075f6-f6a1-475d-b5cb-e8995bff1f09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:16:53.383082 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e075f6-f6a1-475d-b5cb-e8995bff1f09 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:16:53.402750 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 743/1509] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:16:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:16:53.868832 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80baf0a-2555-496a-b8b8-1b51aa9e496f None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:16:53.869892 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-e80baf0a-2555-496a-b8b8-1b51aa9e496f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:16:53.882165 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80baf0a-2555-496a-b8b8-1b51aa9e496f None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:16:53.979392 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a50e6b-7324-4812-996c-3cacf0950eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:16:53.980075 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a50e6b-7324-4812-996c-3cacf0950eba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:16:53.980356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a50e6b-7324-4812-996c-3cacf0950eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:16:53.980631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a50e6b-7324-4812-996c-3cacf0950eba admin admin] PATH_INFO: `/v3/projects/31525ee21bb04fcbbec183fee166a966` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:16:53.991293 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a50e6b-7324-4812-996c-3cacf0950eba admin admin] RBAC: Authorizing `identity:delete_project(project_id=31525ee21bb04fcbbec183fee166a966)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:16:53.992380 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: WARNING py.warnings [None req-d4a50e6b-7324-4812-996c-3cacf0950eba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 12:16:53.992380 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: warnings.warn(msg) Aug 18 12:16:53.992380 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: Aug 18 12:16:53.993604 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a50e6b-7324-4812-996c-3cacf0950eba admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:16:54.104850 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-d4a50e6b-7324-4812-996c-3cacf0950eba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31525ee21bb04fcbbec183fee166a966', 'request_id': 'req-d4a50e6b-7324-4812-996c-3cacf0950eba'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:16:54.110781 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-d4a50e6b-7324-4812-996c-3cacf0950eba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31525ee21bb04fcbbec183fee166a966', 'request_id': 'req-d4a50e6b-7324-4812-996c-3cacf0950eba'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:16:54.191789 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-d4a50e6b-7324-4812-996c-3cacf0950eba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31525ee21bb04fcbbec183fee166a966', 'request_id': 'req-d4a50e6b-7324-4812-996c-3cacf0950eba'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:16:54.222234 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-d4a50e6b-7324-4812-996c-3cacf0950eba admin admin] The token cache is being invalidate because project 31525ee21bb04fcbbec183fee166a966 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:16:54.222521 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-d4a50e6b-7324-4812-996c-3cacf0950eba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:16:54.223053 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-d4a50e6b-7324-4812-996c-3cacf0950eba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:16:54.224987 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 767/1510] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:16:53 2022] DELETE /identity/v3/projects/31525ee21bb04fcbbec183fee166a966 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:17:03.399999 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=62 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:17:04.220329 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:18:22.481855 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bc1c5-8b56-4d35-a830-c04f695afcd6 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:18:22.482469 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bc1c5-8b56-4d35-a830-c04f695afcd6 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:18:22.482469 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bc1c5-8b56-4d35-a830-c04f695afcd6 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:18:22.491298 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 744/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 12:18:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:18:22.491398 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799c565-abed-4a89-a776-f9e2eb1889c9 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:18:22.491398 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799c565-abed-4a89-a776-f9e2eb1889c9 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:18:22.491398 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799c565-abed-4a89-a776-f9e2eb1889c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:18:22.583221 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-a799c565-abed-4a89-a776-f9e2eb1889c9 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:18:22.584481 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-a799c565-abed-4a89-a776-f9e2eb1889c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:18:22.691994 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 768/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 12:18:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:18:22.851340 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1bc1c5-8b56-4d35-a830-c04f695afcd6 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:18:22.853560 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-6e1bc1c5-8b56-4d35-a830-c04f695afcd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:18:22.977805 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1bc1c5-8b56-4d35-a830-c04f695afcd6 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:18:22.984499 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf55f8b-f5b4-42e5-b547-e8a88e707d20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:18:22.992334 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf55f8b-f5b4-42e5-b547-e8a88e707d20 service nova] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:18:22.992334 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf55f8b-f5b4-42e5-b547-e8a88e707d20 service nova] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:18:22.992334 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf55f8b-f5b4-42e5-b547-e8a88e707d20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:18:22.994208 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf55f8b-f5b4-42e5-b547-e8a88e707d20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:18:22.996664 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf55f8b-f5b4-42e5-b547-e8a88e707d20 service nova] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:18:23.006675 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 745/1513] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:18:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:18:23.844888 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155c036-3960-44a1-9e77-58b03d14fa45 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:18:23.845549 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155c036-3960-44a1-9e77-58b03d14fa45 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:18:23.845549 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155c036-3960-44a1-9e77-58b03d14fa45 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:18:23.846836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 769/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 12:18:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:18:23.851994 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23eaef-5cf8-451d-838c-cfecd1aa423f None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:18:23.852539 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23eaef-5cf8-451d-838c-cfecd1aa423f None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:18:23.852539 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23eaef-5cf8-451d-838c-cfecd1aa423f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:18:23.872881 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-ca23eaef-5cf8-451d-838c-cfecd1aa423f None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:18:23.881296 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-ca23eaef-5cf8-451d-838c-cfecd1aa423f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:18:23.882836 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 746/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 12:18:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:18:24.037096 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155c036-3960-44a1-9e77-58b03d14fa45 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:18:24.052303 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155c036-3960-44a1-9e77-58b03d14fa45 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:18:24.058730 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70421176-9b74-4f04-855d-9ffeb3f67936 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:18:24.059331 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70421176-9b74-4f04-855d-9ffeb3f67936 service nova] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:18:24.059672 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70421176-9b74-4f04-855d-9ffeb3f67936 service nova] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:18:24.059952 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70421176-9b74-4f04-855d-9ffeb3f67936 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:18:24.068919 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70421176-9b74-4f04-855d-9ffeb3f67936 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:18:24.070549 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70421176-9b74-4f04-855d-9ffeb3f67936 service nova] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:18:24.081495 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 770/1516] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:18:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:18:24.378995 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fecdff-52b5-4e2d-a316-1b31e40c140d None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:18:24.379294 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fecdff-52b5-4e2d-a316-1b31e40c140d None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:18:24.379612 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fecdff-52b5-4e2d-a316-1b31e40c140d None None] PATH_INFO: `/v3` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:18:24.380894 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 747/1517] 10.222.0.76 () {54 vars in 925 bytes} [Thu Aug 18 12:18:24 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:18:24.387441 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45f11e-93de-479c-9ee9-38115051ab64 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:18:24.387738 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45f11e-93de-479c-9ee9-38115051ab64 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:18:24.388030 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45f11e-93de-479c-9ee9-38115051ab64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:18:24.453536 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-aa45f11e-93de-479c-9ee9-38115051ab64 None None] MFA Rules not processed for user `193637acadc14ec4a48ab79566ba3dab`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:18:24.454764 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-aa45f11e-93de-479c-9ee9-38115051ab64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:18:24.471238 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:18:24.525733 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 771/1518] 10.222.0.76 () {58 vars in 1015 bytes} [Thu Aug 18 12:18:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:18:28.485936 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3348f11-90ac-4878-912f-d851f4478859 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:18:28.486512 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3348f11-90ac-4878-912f-d851f4478859 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:18:28.486512 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3348f11-90ac-4878-912f-d851f4478859 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:18:28.487825 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 748/1519] 10.222.0.98 () {54 vars in 961 bytes} [Thu Aug 18 12:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:18:28.492909 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af97ba-f1d8-420e-99d7-151f874d1568 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:18:28.493346 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af97ba-f1d8-420e-99d7-151f874d1568 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:18:28.493505 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af97ba-f1d8-420e-99d7-151f874d1568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:18:28.513373 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-f4af97ba-f1d8-420e-99d7-151f874d1568 None None] MFA Rules not processed for user `193637acadc14ec4a48ab79566ba3dab`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:18:28.514580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-f4af97ba-f1d8-420e-99d7-151f874d1568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:18:28.523048 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 772/1520] 10.222.0.98 () {58 vars in 1058 bytes} [Thu Aug 18 12:18:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:18:28.530663 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3348f11-90ac-4878-912f-d851f4478859 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:18:28.544315 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3348f11-90ac-4878-912f-d851f4478859 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:18:28.549290 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43245806-da6b-497e-a6e4-9421dcf55b05 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193637acadc14ec4a48ab79566ba3dab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:18:28.549884 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43245806-da6b-497e-a6e4-9421dcf55b05 service placement] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:18:28.550169 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43245806-da6b-497e-a6e4-9421dcf55b05 service placement] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:18:28.550448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43245806-da6b-497e-a6e4-9421dcf55b05 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:18:28.559353 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43245806-da6b-497e-a6e4-9421dcf55b05 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:18:28.560956 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43245806-da6b-497e-a6e4-9421dcf55b05 service placement] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:18:28.570466 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 749/1521] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:18:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:18:35.324654 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39584e1-9847-470b-85e9-c3efac2dd818 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:18:35.325846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39584e1-9847-470b-85e9-c3efac2dd818 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:18:35.325846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39584e1-9847-470b-85e9-c3efac2dd818 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:18:35.327032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 773/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 12:18:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:18:35.331907 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6b073-3bf7-4a16-8cb0-3b536f8a0ab3 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:18:35.332306 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6b073-3bf7-4a16-8cb0-3b536f8a0ab3 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:18:35.332496 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6b073-3bf7-4a16-8cb0-3b536f8a0ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:18:35.351959 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-51e6b073-3bf7-4a16-8cb0-3b536f8a0ab3 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:18:35.353262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-51e6b073-3bf7-4a16-8cb0-3b536f8a0ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:18:35.361262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 750/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 12:18:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:18:35.516396 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39584e1-9847-470b-85e9-c3efac2dd818 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:18:35.530923 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39584e1-9847-470b-85e9-c3efac2dd818 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:18:35.536540 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cce9e0-617e-41a5-b41b-cd82a4dc86fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:18:35.537181 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cce9e0-617e-41a5-b41b-cd82a4dc86fc service nova] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:18:35.537468 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cce9e0-617e-41a5-b41b-cd82a4dc86fc service nova] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:18:35.537761 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cce9e0-617e-41a5-b41b-cd82a4dc86fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:18:35.546164 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cce9e0-617e-41a5-b41b-cd82a4dc86fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:18:35.556860 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cce9e0-617e-41a5-b41b-cd82a4dc86fc service nova] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:18:35.567483 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 774/1524] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:18:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:18:36.391761 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ae7a9-5163-482f-9ffe-e8241b39aa9d None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:18:36.392325 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ae7a9-5163-482f-9ffe-e8241b39aa9d None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:18:36.392325 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ae7a9-5163-482f-9ffe-e8241b39aa9d None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:18:36.393716 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 751/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 18 12:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:18:36.398577 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4dc367-de36-4b18-8786-0c319488b768 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:18:36.399055 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4dc367-de36-4b18-8786-0c319488b768 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:18:36.399215 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4dc367-de36-4b18-8786-0c319488b768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:18:36.418090 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-bb4dc367-de36-4b18-8786-0c319488b768 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:18:36.419791 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-bb4dc367-de36-4b18-8786-0c319488b768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:18:36.427841 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 775/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 18 12:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:18:36.583465 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ae7a9-5163-482f-9ffe-e8241b39aa9d None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:18:36.597657 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ae7a9-5163-482f-9ffe-e8241b39aa9d None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:18:36.603066 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0784bc-f76b-4ac4-b770-7a69c5133731 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:18:36.603684 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0784bc-f76b-4ac4-b770-7a69c5133731 service nova] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:18:36.603962 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0784bc-f76b-4ac4-b770-7a69c5133731 service nova] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:18:36.604234 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0784bc-f76b-4ac4-b770-7a69c5133731 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:18:36.612976 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0784bc-f76b-4ac4-b770-7a69c5133731 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:18:36.614575 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0784bc-f76b-4ac4-b770-7a69c5133731 service nova] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:18:36.624895 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 752/1527] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:18:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:18:40.513267 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e596ac5-e809-4d2e-8f88-3adb2a5a10a9 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:18:40.514481 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e596ac5-e809-4d2e-8f88-3adb2a5a10a9 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:18:40.514481 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e596ac5-e809-4d2e-8f88-3adb2a5a10a9 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:18:40.515825 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 776/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 12:18:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:18:40.520954 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44e066a-57b1-4bc4-a5e8-4e3abf62cc04 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:18:40.521382 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44e066a-57b1-4bc4-a5e8-4e3abf62cc04 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:18:40.521493 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44e066a-57b1-4bc4-a5e8-4e3abf62cc04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:18:40.540374 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-b44e066a-57b1-4bc4-a5e8-4e3abf62cc04 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:18:40.541570 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-b44e066a-57b1-4bc4-a5e8-4e3abf62cc04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:18:40.549379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 753/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 12:18:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:18:40.831357 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e596ac5-e809-4d2e-8f88-3adb2a5a10a9 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:18:40.833109 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-0e596ac5-e809-4d2e-8f88-3adb2a5a10a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:18:40.912209 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e596ac5-e809-4d2e-8f88-3adb2a5a10a9 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:18:40.915859 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc46920a-932b-49f9-8dbe-5ceb6038f58d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:18:40.916471 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46920a-932b-49f9-8dbe-5ceb6038f58d service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:18:40.916756 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46920a-932b-49f9-8dbe-5ceb6038f58d service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:18:40.917037 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46920a-932b-49f9-8dbe-5ceb6038f58d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:18:40.927803 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc46920a-932b-49f9-8dbe-5ceb6038f58d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:18:40.929255 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc46920a-932b-49f9-8dbe-5ceb6038f58d service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:18:40.939762 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 777/1530] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:18:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:18:42.838797 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7a014-a6c1-44c2-a45c-c05bf5d98e9d None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:18:42.839526 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7a014-a6c1-44c2-a45c-c05bf5d98e9d None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:18:42.839526 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7a014-a6c1-44c2-a45c-c05bf5d98e9d None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:18:42.840746 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 754/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 12:18:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 18 12:18:42.846739 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b41d4e-be67-4adb-af33-022fe44b4ddd None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:18:42.847231 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b41d4e-be67-4adb-af33-022fe44b4ddd None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:18:42.847348 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b41d4e-be67-4adb-af33-022fe44b4ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:18:42.867359 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-a0b41d4e-be67-4adb-af33-022fe44b4ddd None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:18:42.868598 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-a0b41d4e-be67-4adb-af33-022fe44b4ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:18:42.876760 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 778/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 12:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:18:43.166463 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a7a014-a6c1-44c2-a45c-c05bf5d98e9d None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:18:43.177094 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a7a014-a6c1-44c2-a45c-c05bf5d98e9d None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:18:43.180574 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77292c72-9398-440e-bc1a-8e6a9ccf2fc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:18:43.181158 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77292c72-9398-440e-bc1a-8e6a9ccf2fc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:18:43.181436 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77292c72-9398-440e-bc1a-8e6a9ccf2fc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:18:43.181706 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77292c72-9398-440e-bc1a-8e6a9ccf2fc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:18:43.190292 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77292c72-9398-440e-bc1a-8e6a9ccf2fc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:18:43.191743 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77292c72-9398-440e-bc1a-8e6a9ccf2fc1 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:18:43.201799 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 755/1533] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:18:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:18:46.722376 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61bbbd-5fdf-4644-9395-c00ebb0d5a31 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:18:46.724541 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61bbbd-5fdf-4644-9395-c00ebb0d5a31 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:18:46.724541 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61bbbd-5fdf-4644-9395-c00ebb0d5a31 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:18:46.724541 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 779/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 12:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:18:46.729459 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f462b7a-2476-4c3b-a848-54ecf0f7ae02 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:18:46.729873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f462b7a-2476-4c3b-a848-54ecf0f7ae02 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:18:46.730000 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f462b7a-2476-4c3b-a848-54ecf0f7ae02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:18:46.737601 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=81 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:18:46.750007 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-0f462b7a-2476-4c3b-a848-54ecf0f7ae02 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:18:46.751513 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-0f462b7a-2476-4c3b-a848-54ecf0f7ae02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:18:46.759482 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 756/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 12:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:18:47.315151 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61bbbd-5fdf-4644-9395-c00ebb0d5a31 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:18:47.316885 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-1a61bbbd-5fdf-4644-9395-c00ebb0d5a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:18:47.380951 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:18:47.394616 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61bbbd-5fdf-4644-9395-c00ebb0d5a31 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:18:47.398321 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2080b53-59f5-423d-8eb3-5f02769477b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:18:47.398951 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2080b53-59f5-423d-8eb3-5f02769477b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:18:47.401702 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2080b53-59f5-423d-8eb3-5f02769477b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:18:47.401702 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2080b53-59f5-423d-8eb3-5f02769477b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:18:47.409499 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2080b53-59f5-423d-8eb3-5f02769477b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:18:47.410944 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2080b53-59f5-423d-8eb3-5f02769477b9 service cinder] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:18:47.421801 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 780/1536] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:18:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:18:49.362038 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1baee01-733a-4c3c-8eda-821d8ba864ff None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:18:49.362593 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1baee01-733a-4c3c-8eda-821d8ba864ff None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:18:49.362593 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1baee01-733a-4c3c-8eda-821d8ba864ff None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:18:49.363870 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 757/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 18 12:18:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:18:49.369067 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b37dcc-77c8-423c-9e5d-2b4e4133e0dd None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:18:49.369485 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b37dcc-77c8-423c-9e5d-2b4e4133e0dd None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:18:49.369616 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b37dcc-77c8-423c-9e5d-2b4e4133e0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:18:49.390830 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-d3b37dcc-77c8-423c-9e5d-2b4e4133e0dd None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:18:49.392094 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-d3b37dcc-77c8-423c-9e5d-2b4e4133e0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:18:49.400465 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 781/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 18 12:18:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:18:49.957845 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1baee01-733a-4c3c-8eda-821d8ba864ff None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:18:49.959543 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-d1baee01-733a-4c3c-8eda-821d8ba864ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:18:49.973094 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1baee01-733a-4c3c-8eda-821d8ba864ff None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:18:49.976638 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6452d-c957-4d5e-b0de-bf854fca0042 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:18:49.977236 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6452d-c957-4d5e-b0de-bf854fca0042 service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:18:49.977520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6452d-c957-4d5e-b0de-bf854fca0042 service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:18:49.977793 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6452d-c957-4d5e-b0de-bf854fca0042 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:18:49.986630 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed6452d-c957-4d5e-b0de-bf854fca0042 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:18:49.988085 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed6452d-c957-4d5e-b0de-bf854fca0042 service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:18:49.997494 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 758/1539] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:18:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:18:59.385145 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=11 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:18:59.993283 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=8 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:13.948871 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11ed91-2122-4c3c-b1b5-8bb2e635e943 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:13.949493 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11ed91-2122-4c3c-b1b5-8bb2e635e943 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:13.949493 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11ed91-2122-4c3c-b1b5-8bb2e635e943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:13.974924 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-7f11ed91-2122-4c3c-b1b5-8bb2e635e943 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:13.976189 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-7f11ed91-2122-4c3c-b1b5-8bb2e635e943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:13.977687 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b647fe-c442-489e-9da5-1cffd3284dab None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:13.978202 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b647fe-c442-489e-9da5-1cffd3284dab None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:13.978885 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b647fe-c442-489e-9da5-1cffd3284dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:13.990532 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 782/1540] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:13.999115 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77de14-76a1-4ce3-b7ee-0695bc18d851 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:13.999459 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77de14-76a1-4ce3-b7ee-0695bc18d851 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:13.999707 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77de14-76a1-4ce3-b7ee-0695bc18d851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:14.009813 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-48b647fe-c442-489e-9da5-1cffd3284dab None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:14.011282 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-48b647fe-c442-489e-9da5-1cffd3284dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:14.023841 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 759/1541] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:14.025983 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-df77de14-76a1-4ce3-b7ee-0695bc18d851 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:14.027211 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-df77de14-76a1-4ce3-b7ee-0695bc18d851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:14.032542 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7615b-8ca0-4d27-921e-ae07373cd76f None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:14.032839 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7615b-8ca0-4d27-921e-ae07373cd76f None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:14.033102 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7615b-8ca0-4d27-921e-ae07373cd76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:14.042107 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 783/1542] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:20:14.043383 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b1a5a-d7db-4736-a450-534ac2885466 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:14.043699 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b1a5a-d7db-4736-a450-534ac2885466 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:14.043980 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b1a5a-d7db-4736-a450-534ac2885466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:14.068791 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-69f7615b-8ca0-4d27-921e-ae07373cd76f None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:14.070041 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-69f7615b-8ca0-4d27-921e-ae07373cd76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:14.076325 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-d85b1a5a-d7db-4736-a450-534ac2885466 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:14.077553 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-d85b1a5a-d7db-4736-a450-534ac2885466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:14.086188 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 760/1543] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:20:14.087214 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f7615b-8ca0-4d27-921e-ae07373cd76f None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:14.102557 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f7615b-8ca0-4d27-921e-ae07373cd76f None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:14.112814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f98a025-d98d-4363-b7c8-6779e0f05698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:14.113718 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98a025-d98d-4363-b7c8-6779e0f05698 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:14.114186 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98a025-d98d-4363-b7c8-6779e0f05698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:14.114630 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98a025-d98d-4363-b7c8-6779e0f05698 admin admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:14.115690 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f98a025-d98d-4363-b7c8-6779e0f05698 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:20:14.116178 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f98a025-d98d-4363-b7c8-6779e0f05698 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:14.117809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: WARNING py.warnings [None req-7f98a025-d98d-4363-b7c8-6779e0f05698 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 18 12:20:14.117809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: warnings.warn(msg) Aug 18 12:20:14.117809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: Aug 18 12:20:14.119195 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f98a025-d98d-4363-b7c8-6779e0f05698 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:14.120533 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 784/1544] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:14.121417 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b1a5a-d7db-4736-a450-534ac2885466 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:14.141373 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b1a5a-d7db-4736-a450-534ac2885466 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:14.142327 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 761/1545] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:20:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:20:14.143666 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15361e6b-1ccf-4079-8c2b-3bf9a27e0964 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:14.143956 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15361e6b-1ccf-4079-8c2b-3bf9a27e0964 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:14.144233 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15361e6b-1ccf-4079-8c2b-3bf9a27e0964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:14.154306 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c81ca2f-b8af-4564-b3a3-542b324a5de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:14.154989 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81ca2f-b8af-4564-b3a3-542b324a5de1 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:14.155276 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81ca2f-b8af-4564-b3a3-542b324a5de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:14.155578 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81ca2f-b8af-4564-b3a3-542b324a5de1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:14.158007 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c81ca2f-b8af-4564-b3a3-542b324a5de1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:20:14.158320 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c81ca2f-b8af-4564-b3a3-542b324a5de1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:14.159972 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c81ca2f-b8af-4564-b3a3-542b324a5de1 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:14.175287 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-15361e6b-1ccf-4079-8c2b-3bf9a27e0964 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:14.179595 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 785/1546] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:20:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:20:14.179595 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166f8b7-88ca-4c70-a3b9-4347a76c80a2 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:14.179595 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166f8b7-88ca-4c70-a3b9-4347a76c80a2 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:14.179595 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166f8b7-88ca-4c70-a3b9-4347a76c80a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:14.179995 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-15361e6b-1ccf-4079-8c2b-3bf9a27e0964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:14.192117 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 762/1547] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:14.193544 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4835259-71ff-49c9-a62b-f99ded53b94e None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:14.193833 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4835259-71ff-49c9-a62b-f99ded53b94e None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:14.194106 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4835259-71ff-49c9-a62b-f99ded53b94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:14.204705 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-1166f8b7-88ca-4c70-a3b9-4347a76c80a2 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:14.206034 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-1166f8b7-88ca-4c70-a3b9-4347a76c80a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:14.214470 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 786/1548] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:14.215815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d05bd-d296-4449-aa5f-7f656f6b6142 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:14.216178 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d05bd-d296-4449-aa5f-7f656f6b6142 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:14.216530 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d05bd-d296-4449-aa5f-7f656f6b6142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:14.217080 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-a4835259-71ff-49c9-a62b-f99ded53b94e None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:14.220080 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-a4835259-71ff-49c9-a62b-f99ded53b94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:14.230681 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 763/1549] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:14.231569 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4835259-71ff-49c9-a62b-f99ded53b94e None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:14.238278 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-e30d05bd-d296-4449-aa5f-7f656f6b6142 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:14.239625 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-e30d05bd-d296-4449-aa5f-7f656f6b6142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:14.242398 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4835259-71ff-49c9-a62b-f99ded53b94e None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:14.246871 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e03f1c1-1cd5-44f6-9d75-206ae5ee760a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:14.247504 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03f1c1-1cd5-44f6-9d75-206ae5ee760a admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:14.248042 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03f1c1-1cd5-44f6-9d75-206ae5ee760a admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:14.248267 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 787/1550] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:14.248373 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03f1c1-1cd5-44f6-9d75-206ae5ee760a admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:14.248927 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e03f1c1-1cd5-44f6-9d75-206ae5ee760a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:14.249545 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee24e52-ee86-4652-8633-db828fc00815 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:14.249867 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee24e52-ee86-4652-8633-db828fc00815 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:14.250145 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee24e52-ee86-4652-8633-db828fc00815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:14.250956 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e03f1c1-1cd5-44f6-9d75-206ae5ee760a admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:14.270091 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-7ee24e52-ee86-4652-8633-db828fc00815 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:14.271293 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-7ee24e52-ee86-4652-8633-db828fc00815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:14.281537 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 788/1551] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:14.282313 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee24e52-ee86-4652-8633-db828fc00815 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:14.291789 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee24e52-ee86-4652-8633-db828fc00815 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:14.296265 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa01aa5-c4f3-43d2-8e51-4f9cd1794c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:14.297517 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa01aa5-c4f3-43d2-8e51-4f9cd1794c74 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:14.297794 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa01aa5-c4f3-43d2-8e51-4f9cd1794c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:14.298063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa01aa5-c4f3-43d2-8e51-4f9cd1794c74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:14.298674 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa01aa5-c4f3-43d2-8e51-4f9cd1794c74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:14.299730 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: WARNING py.warnings [None req-faa01aa5-c4f3-43d2-8e51-4f9cd1794c74 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 12:20:14.299730 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: warnings.warn(msg) Aug 18 12:20:14.299730 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: Aug 18 12:20:14.300908 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa01aa5-c4f3-43d2-8e51-4f9cd1794c74 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:14.365605 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 764/1552] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:20:14 2022] POST /identity/v3/projects => generated 403 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:14.365605 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51484914-9cb8-439c-bd34-d2db96fd4f5e None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:14.365605 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51484914-9cb8-439c-bd34-d2db96fd4f5e None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:14.365605 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51484914-9cb8-439c-bd34-d2db96fd4f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:14.386987 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-51484914-9cb8-439c-bd34-d2db96fd4f5e None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:14.388316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 789/1553] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:20:14 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:14.388474 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-51484914-9cb8-439c-bd34-d2db96fd4f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:14.389635 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704a021-6dcd-4d45-bec1-9716fd395429 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:14.389927 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704a021-6dcd-4d45-bec1-9716fd395429 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:14.390268 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704a021-6dcd-4d45-bec1-9716fd395429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:14.410587 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-d704a021-6dcd-4d45-bec1-9716fd395429 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:14.416986 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-d704a021-6dcd-4d45-bec1-9716fd395429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:14.462586 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 765/1554] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:14.462586 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51484914-9cb8-439c-bd34-d2db96fd4f5e None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:14.470579 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51484914-9cb8-439c-bd34-d2db96fd4f5e None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:14.472198 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 790/1555] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:14.473467 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093f821-07e9-4a7d-85df-805ebc433718 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:14.473759 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093f821-07e9-4a7d-85df-805ebc433718 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:14.474100 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093f821-07e9-4a7d-85df-805ebc433718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:14.477217 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe8ee4-d5df-49f8-9fb1-f9e99ac67653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:14.479990 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe8ee4-d5df-49f8-9fb1-f9e99ac67653 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:14.479990 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe8ee4-d5df-49f8-9fb1-f9e99ac67653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:14.479990 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe8ee4-d5df-49f8-9fb1-f9e99ac67653 admin admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:14.479990 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe8ee4-d5df-49f8-9fb1-f9e99ac67653 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:20:14.479990 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe8ee4-d5df-49f8-9fb1-f9e99ac67653 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:14.480969 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe8ee4-d5df-49f8-9fb1-f9e99ac67653 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:14.493170 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 766/1556] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:20:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:20:14.494482 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900fd4bd-4741-47f3-abb7-959bb84a87bf None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:14.494770 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900fd4bd-4741-47f3-abb7-959bb84a87bf None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:14.495043 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900fd4bd-4741-47f3-abb7-959bb84a87bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:14.503600 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-b093f821-07e9-4a7d-85df-805ebc433718 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:14.505055 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-b093f821-07e9-4a7d-85df-805ebc433718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:14.513266 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 791/1557] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:14.517918 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1544e0d-5edc-4c2b-80dc-d94d96633242 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:14.517918 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1544e0d-5edc-4c2b-80dc-d94d96633242 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:14.517918 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1544e0d-5edc-4c2b-80dc-d94d96633242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:14.518206 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-900fd4bd-4741-47f3-abb7-959bb84a87bf None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:14.518206 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-900fd4bd-4741-47f3-abb7-959bb84a87bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:14.526028 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 767/1558] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:14.526867 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900fd4bd-4741-47f3-abb7-959bb84a87bf None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:14.536229 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-a1544e0d-5edc-4c2b-80dc-d94d96633242 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:14.536739 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900fd4bd-4741-47f3-abb7-959bb84a87bf None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:14.537430 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-a1544e0d-5edc-4c2b-80dc-d94d96633242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:14.541385 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4eb0d-783a-4fea-a48a-457f8c688786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:14.541984 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4eb0d-783a-4fea-a48a-457f8c688786 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:14.542263 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4eb0d-783a-4fea-a48a-457f8c688786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:14.542537 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4eb0d-783a-4fea-a48a-457f8c688786 admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:14.543156 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef4eb0d-783a-4fea-a48a-457f8c688786 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:14.544214 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: WARNING py.warnings [None req-7ef4eb0d-783a-4fea-a48a-457f8c688786 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 12:20:14.544214 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: warnings.warn(msg) Aug 18 12:20:14.544214 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: Aug 18 12:20:14.545470 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 792/1559] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:14.545598 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef4eb0d-783a-4fea-a48a-457f8c688786 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:14.546074 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1544e0d-5edc-4c2b-80dc-d94d96633242 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:14.556297 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1544e0d-5edc-4c2b-80dc-d94d96633242 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:14.561356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdac98d-9db0-43c3-b2b8-01bbdfdb3462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:14.561960 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdac98d-9db0-43c3-b2b8-01bbdfdb3462 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:14.562300 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdac98d-9db0-43c3-b2b8-01bbdfdb3462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:14.562529 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdac98d-9db0-43c3-b2b8-01bbdfdb3462 admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:14.563202 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdac98d-9db0-43c3-b2b8-01bbdfdb3462 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:14.565245 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdac98d-9db0-43c3-b2b8-01bbdfdb3462 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:14.620937 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 768/1560] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:20:14 2022] POST /identity/v3/users => generated 302 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:14.621768 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4eb0d-783a-4fea-a48a-457f8c688786 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:14.631556 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4eb0d-783a-4fea-a48a-457f8c688786 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:14.636229 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2b0ce-6fa9-4151-82ea-7ce3444eb33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:14.636859 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2b0ce-6fa9-4151-82ea-7ce3444eb33e admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:14.637140 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2b0ce-6fa9-4151-82ea-7ce3444eb33e admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:14.637419 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2b0ce-6fa9-4151-82ea-7ce3444eb33e admin admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:14.638055 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e2b0ce-6fa9-4151-82ea-7ce3444eb33e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:20:14.638357 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e2b0ce-6fa9-4151-82ea-7ce3444eb33e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:14.639970 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e2b0ce-6fa9-4151-82ea-7ce3444eb33e admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:14.651454 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 793/1561] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:20:14 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:14.651454 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a6e92-63ae-4c47-a1db-ae9cce54df5a None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:14.661476 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a6e92-63ae-4c47-a1db-ae9cce54df5a None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:14.661476 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a6e92-63ae-4c47-a1db-ae9cce54df5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:14.661665 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 769/1562] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:20:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:20:14.661665 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2b0ce-6fa9-4151-82ea-7ce3444eb33e admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:14.667678 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2b0ce-6fa9-4151-82ea-7ce3444eb33e admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:14.672526 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024e4eb-7129-43c5-a93b-a2ac9872f895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:14.673149 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024e4eb-7129-43c5-a93b-a2ac9872f895 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:14.673432 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024e4eb-7129-43c5-a93b-a2ac9872f895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:14.673711 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024e4eb-7129-43c5-a93b-a2ac9872f895 admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:14.674344 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024e4eb-7129-43c5-a93b-a2ac9872f895 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:14.676296 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024e4eb-7129-43c5-a93b-a2ac9872f895 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:14.681192 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-c80a6e92-63ae-4c47-a1db-ae9cce54df5a None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:14.690225 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-c80a6e92-63ae-4c47-a1db-ae9cce54df5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:14.691088 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 794/1563] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:14.692166 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a6e92-63ae-4c47-a1db-ae9cce54df5a None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:14.703823 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a6e92-63ae-4c47-a1db-ae9cce54df5a None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:14.708601 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eb953b-e868-49f8-8351-8bc79eeccab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:14.709219 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb953b-e868-49f8-8351-8bc79eeccab1 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:14.719336 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb953b-e868-49f8-8351-8bc79eeccab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:14.719336 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb953b-e868-49f8-8351-8bc79eeccab1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:14.719336 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eb953b-e868-49f8-8351-8bc79eeccab1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:20:14.719336 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eb953b-e868-49f8-8351-8bc79eeccab1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:14.719336 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eb953b-e868-49f8-8351-8bc79eeccab1 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:14.724841 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 795/1564] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:20:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:20:14.725660 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eb953b-e868-49f8-8351-8bc79eeccab1 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:14.735341 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eb953b-e868-49f8-8351-8bc79eeccab1 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:14.740034 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c448516-6191-4b07-8579-dbe4f725cdac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:14.740629 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c448516-6191-4b07-8579-dbe4f725cdac admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:14.740922 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c448516-6191-4b07-8579-dbe4f725cdac admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:14.741197 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c448516-6191-4b07-8579-dbe4f725cdac admin admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:14.741841 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c448516-6191-4b07-8579-dbe4f725cdac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:20:14.742147 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c448516-6191-4b07-8579-dbe4f725cdac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:14.743771 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c448516-6191-4b07-8579-dbe4f725cdac admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:14.756232 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 796/1565] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:20:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:20:14.757011 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c448516-6191-4b07-8579-dbe4f725cdac admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:14.767078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c448516-6191-4b07-8579-dbe4f725cdac admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:14.771770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2577813d-e155-4d56-a412-ee33b4d63f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:14.772445 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2577813d-e155-4d56-a412-ee33b4d63f09 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:14.772645 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2577813d-e155-4d56-a412-ee33b4d63f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:14.772926 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2577813d-e155-4d56-a412-ee33b4d63f09 admin admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:14.773706 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2577813d-e155-4d56-a412-ee33b4d63f09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:20:14.773874 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2577813d-e155-4d56-a412-ee33b4d63f09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:14.775481 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2577813d-e155-4d56-a412-ee33b4d63f09 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:14.786509 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 770/1566] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:20:14 2022] POST /identity/v3/projects => generated 367 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:14.786509 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024e4eb-7129-43c5-a93b-a2ac9872f895 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:14.788975 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 797/1567] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:20:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:20:14.789724 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2577813d-e155-4d56-a412-ee33b4d63f09 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:14.799574 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2577813d-e155-4d56-a412-ee33b4d63f09 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:14.805304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:14.808990 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:14.851853 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024e4eb-7129-43c5-a93b-a2ac9872f895 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:14.856941 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea4f4d-cbf6-443b-83be-74fa09cd75f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:14.857561 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea4f4d-cbf6-443b-83be-74fa09cd75f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:14.858011 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea4f4d-cbf6-443b-83be-74fa09cd75f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:14.858128 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea4f4d-cbf6-443b-83be-74fa09cd75f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:14.858759 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea4f4d-cbf6-443b-83be-74fa09cd75f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:14.860375 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1280c4c5-e033-4c47-9621-439e6e0dd193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:14.864236 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea4f4d-cbf6-443b-83be-74fa09cd75f3 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:14.864350 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280c4c5-e033-4c47-9621-439e6e0dd193 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:14.864350 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280c4c5-e033-4c47-9621-439e6e0dd193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:14.864350 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280c4c5-e033-4c47-9621-439e6e0dd193 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:14.864350 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1280c4c5-e033-4c47-9621-439e6e0dd193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:14.864350 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1280c4c5-e033-4c47-9621-439e6e0dd193 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:14.870494 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 771/1568] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:14 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:14.872363 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea4f4d-cbf6-443b-83be-74fa09cd75f3 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:14.875316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 798/1569] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:14 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:14.876422 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1280c4c5-e033-4c47-9621-439e6e0dd193 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:14.884092 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea4f4d-cbf6-443b-83be-74fa09cd75f3 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:14.886937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1280c4c5-e033-4c47-9621-439e6e0dd193 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:14.888890 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae84eb-04f3-4201-82bb-b7d056457618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:14.892931 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae84eb-04f3-4201-82bb-b7d056457618 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:14.892931 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae84eb-04f3-4201-82bb-b7d056457618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:14.892931 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae84eb-04f3-4201-82bb-b7d056457618 admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:14.892931 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fae84eb-04f3-4201-82bb-b7d056457618 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:14.893333 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994c33c-71b5-4993-b31c-073cfb4f02ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:14.893333 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994c33c-71b5-4993-b31c-073cfb4f02ca admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:14.893550 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fae84eb-04f3-4201-82bb-b7d056457618 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:14.893649 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994c33c-71b5-4993-b31c-073cfb4f02ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:14.893649 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994c33c-71b5-4993-b31c-073cfb4f02ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:14.894099 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2994c33c-71b5-4993-b31c-073cfb4f02ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:20:14.894409 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2994c33c-71b5-4993-b31c-073cfb4f02ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:14.896018 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2994c33c-71b5-4993-b31c-073cfb4f02ca admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:14.908301 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 799/1570] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:20:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:20:14.909266 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994c33c-71b5-4993-b31c-073cfb4f02ca admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:14.918867 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994c33c-71b5-4993-b31c-073cfb4f02ca admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:14.923691 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b07c4f-b1df-4cec-884f-60b4aafc9090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:14.924359 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b07c4f-b1df-4cec-884f-60b4aafc9090 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:14.924572 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b07c4f-b1df-4cec-884f-60b4aafc9090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:14.924850 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b07c4f-b1df-4cec-884f-60b4aafc9090 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:14.925472 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b07c4f-b1df-4cec-884f-60b4aafc9090 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:14.927444 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b07c4f-b1df-4cec-884f-60b4aafc9090 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:14.982615 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 772/1571] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:20:14 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:14.983364 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae84eb-04f3-4201-82bb-b7d056457618 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:15.045066 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae84eb-04f3-4201-82bb-b7d056457618 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:15.049788 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fcf2a2-4ea3-44ce-8618-964d26d431d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:15.050408 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fcf2a2-4ea3-44ce-8618-964d26d431d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:15.050693 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fcf2a2-4ea3-44ce-8618-964d26d431d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:15.050967 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fcf2a2-4ea3-44ce-8618-964d26d431d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:15.051631 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fcf2a2-4ea3-44ce-8618-964d26d431d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:15.053587 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fcf2a2-4ea3-44ce-8618-964d26d431d6 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:15.066105 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 800/1572] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:20:14 2022] POST /identity/v3/projects => generated 391 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:15.067001 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b07c4f-b1df-4cec-884f-60b4aafc9090 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:15.128747 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b07c4f-b1df-4cec-884f-60b4aafc9090 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:15.133377 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7282c0-b16d-4de3-ba33-bf40e774e0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:15.133981 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7282c0-b16d-4de3-ba33-bf40e774e0c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:15.134264 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7282c0-b16d-4de3-ba33-bf40e774e0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:15.134540 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7282c0-b16d-4de3-ba33-bf40e774e0c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:15.135157 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7282c0-b16d-4de3-ba33-bf40e774e0c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:15.137093 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7282c0-b16d-4de3-ba33-bf40e774e0c7 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:15.159573 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 773/1573] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:20:14 2022] POST /identity/v3/projects => generated 379 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:15.160390 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fcf2a2-4ea3-44ce-8618-964d26d431d6 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:15.215652 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 801/1574] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:20:15 2022] POST /identity/v3/users => generated 284 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:15.216503 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7282c0-b16d-4de3-ba33-bf40e774e0c7 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:15.226175 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7282c0-b16d-4de3-ba33-bf40e774e0c7 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:15.237136 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fcf2a2-4ea3-44ce-8618-964d26d431d6 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:15.242065 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380fdad3-8b72-4b14-9629-eb0255c9d4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:15.242678 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380fdad3-8b72-4b14-9629-eb0255c9d4ed admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:15.242963 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380fdad3-8b72-4b14-9629-eb0255c9d4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:15.243269 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380fdad3-8b72-4b14-9629-eb0255c9d4ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:15.244080 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380fdad3-8b72-4b14-9629-eb0255c9d4ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:15.246017 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380fdad3-8b72-4b14-9629-eb0255c9d4ed admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:15.282977 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d050aa-0b04-4e5f-a9d6-b816e0482c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:15.283621 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d050aa-0b04-4e5f-a9d6-b816e0482c89 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:15.283900 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d050aa-0b04-4e5f-a9d6-b816e0482c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:15.284171 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d050aa-0b04-4e5f-a9d6-b816e0482c89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:15.284809 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d050aa-0b04-4e5f-a9d6-b816e0482c89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:15.286388 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d050aa-0b04-4e5f-a9d6-b816e0482c89 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:15.287616 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:15.296436 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 802/1575] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:15 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:15.297763 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d050aa-0b04-4e5f-a9d6-b816e0482c89 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:15.306948 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d050aa-0b04-4e5f-a9d6-b816e0482c89 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:15.311542 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6fca19-bfdb-45f5-8b55-ccb59ee4b900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:15.312108 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6fca19-bfdb-45f5-8b55-ccb59ee4b900 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:15.312965 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6fca19-bfdb-45f5-8b55-ccb59ee4b900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:15.313240 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6fca19-bfdb-45f5-8b55-ccb59ee4b900 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:15.313857 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6fca19-bfdb-45f5-8b55-ccb59ee4b900 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:15.315407 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6fca19-bfdb-45f5-8b55-ccb59ee4b900 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:15.320855 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 774/1576] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:20:15 2022] POST /identity/v3/projects => generated 373 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:15.321542 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380fdad3-8b72-4b14-9629-eb0255c9d4ed admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:15.325910 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:15.329180 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 803/1577] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:15.329180 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6fca19-bfdb-45f5-8b55-ccb59ee4b900 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:15.338106 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6fca19-bfdb-45f5-8b55-ccb59ee4b900 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:15.388579 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380fdad3-8b72-4b14-9629-eb0255c9d4ed admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:15.393841 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3028bd51-8fb3-42b3-b1ed-79a535ee2b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:15.394444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028bd51-8fb3-42b3-b1ed-79a535ee2b03 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:15.395273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028bd51-8fb3-42b3-b1ed-79a535ee2b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:15.395273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028bd51-8fb3-42b3-b1ed-79a535ee2b03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:15.395659 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3028bd51-8fb3-42b3-b1ed-79a535ee2b03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:15.395967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a01f597-4415-4949-b683-7e042183f596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:15.396656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01f597-4415-4949-b683-7e042183f596 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:15.396865 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01f597-4415-4949-b683-7e042183f596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:15.397142 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01f597-4415-4949-b683-7e042183f596 admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:15.397567 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3028bd51-8fb3-42b3-b1ed-79a535ee2b03 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:15.397786 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a01f597-4415-4949-b683-7e042183f596 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:15.399707 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a01f597-4415-4949-b683-7e042183f596 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:15.490377 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 804/1578] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:20:15 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:15.491223 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a01f597-4415-4949-b683-7e042183f596 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:15.501925 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a01f597-4415-4949-b683-7e042183f596 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:15.506407 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 775/1579] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:20:15 2022] POST /identity/v3/projects => generated 373 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:15.506407 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3028bd51-8fb3-42b3-b1ed-79a535ee2b03 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:15.558194 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c9710-54e8-4ed4-af18-db340302bbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:15.558808 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c9710-54e8-4ed4-af18-db340302bbf2 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:15.559093 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c9710-54e8-4ed4-af18-db340302bbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:15.559375 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c9710-54e8-4ed4-af18-db340302bbf2 admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:15.560036 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860c9710-54e8-4ed4-af18-db340302bbf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:15.561974 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860c9710-54e8-4ed4-af18-db340302bbf2 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:15.565119 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3028bd51-8fb3-42b3-b1ed-79a535ee2b03 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:15.569800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6153288-59d5-4d08-bd65-168e478d11d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:15.570412 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6153288-59d5-4d08-bd65-168e478d11d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:15.570698 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6153288-59d5-4d08-bd65-168e478d11d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:15.570975 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6153288-59d5-4d08-bd65-168e478d11d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:15.571633 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6153288-59d5-4d08-bd65-168e478d11d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:15.573534 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6153288-59d5-4d08-bd65-168e478d11d8 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:15.667042 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 805/1580] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:20:15 2022] POST /identity/v3/users => generated 296 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:15.668153 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c9710-54e8-4ed4-af18-db340302bbf2 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:15.678925 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c9710-54e8-4ed4-af18-db340302bbf2 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:15.683819 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c05094-3815-4b5f-9398-8c762a485392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:15.684426 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c05094-3815-4b5f-9398-8c762a485392 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:15.684711 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c05094-3815-4b5f-9398-8c762a485392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:15.684995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c05094-3815-4b5f-9398-8c762a485392 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:15.685646 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c05094-3815-4b5f-9398-8c762a485392 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:15.687233 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c05094-3815-4b5f-9398-8c762a485392 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:15.703063 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 776/1581] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:20:15 2022] POST /identity/v3/users => generated 290 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:15.703284 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 806/1582] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:15.703284 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c05094-3815-4b5f-9398-8c762a485392 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:15.704317 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6153288-59d5-4d08-bd65-168e478d11d8 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:15.710963 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c05094-3815-4b5f-9398-8c762a485392 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:15.715296 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6153288-59d5-4d08-bd65-168e478d11d8 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:15.715680 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf420419-aba1-4fe6-a465-4b1529ec3578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:15.716284 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf420419-aba1-4fe6-a465-4b1529ec3578 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:15.716572 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf420419-aba1-4fe6-a465-4b1529ec3578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:15.716852 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf420419-aba1-4fe6-a465-4b1529ec3578 admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:15.717481 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf420419-aba1-4fe6-a465-4b1529ec3578 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:15.719950 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf420419-aba1-4fe6-a465-4b1529ec3578 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:15.720793 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb78947-94f9-4bb9-8230-9ff007da3312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:15.721454 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb78947-94f9-4bb9-8230-9ff007da3312 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:15.721734 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb78947-94f9-4bb9-8230-9ff007da3312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:15.722046 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb78947-94f9-4bb9-8230-9ff007da3312 admin admin] PATH_INFO: `/v3/projects/edb4c111c6354e17a9ff18ac60aba626/users/4cb5b947f84549ecb0145907ecceac63/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:15.738575 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb78947-94f9-4bb9-8230-9ff007da3312 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edb4c111c6354e17a9ff18ac60aba626, user_id=4cb5b947f84549ecb0145907ecceac63, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:15.745287 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb78947-94f9-4bb9-8230-9ff007da3312 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:15.847646 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 807/1583] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:20:15 2022] POST /identity/v3/users => generated 287 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:15.847646 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf420419-aba1-4fe6-a465-4b1529ec3578 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:15.854935 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf420419-aba1-4fe6-a465-4b1529ec3578 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:15.860262 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37566e6-76a3-42c9-a8b4-b08a703af3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:15.860901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37566e6-76a3-42c9-a8b4-b08a703af3a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:15.861184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37566e6-76a3-42c9-a8b4-b08a703af3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:15.861597 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37566e6-76a3-42c9-a8b4-b08a703af3a4 admin admin] PATH_INFO: `/v3/projects/296b5d138eff48958150d7dda904e484/users/c3b28d23b0f24ec7acc84fb474e5c721/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:15.873683 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 777/1584] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:20:15 2022] PUT /identity/v3/projects/edb4c111c6354e17a9ff18ac60aba626/users/4cb5b947f84549ecb0145907ecceac63/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:15.883673 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb78947-94f9-4bb9-8230-9ff007da3312 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:15.883820 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37566e6-76a3-42c9-a8b4-b08a703af3a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=296b5d138eff48958150d7dda904e484, user_id=c3b28d23b0f24ec7acc84fb474e5c721, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:15.883820 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: WARNING py.warnings [None req-b37566e6-76a3-42c9-a8b4-b08a703af3a4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 12:20:15.883820 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: warnings.warn(msg) Aug 18 12:20:15.883820 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: Aug 18 12:20:15.883820 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37566e6-76a3-42c9-a8b4-b08a703af3a4 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:15.927009 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:15.937186 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb78947-94f9-4bb9-8230-9ff007da3312 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:15.941848 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b95c51-318c-4337-bf57-8538916fcde0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:15.942458 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b95c51-318c-4337-bf57-8538916fcde0 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:15.942739 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b95c51-318c-4337-bf57-8538916fcde0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:15.943012 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b95c51-318c-4337-bf57-8538916fcde0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:15.943687 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b95c51-318c-4337-bf57-8538916fcde0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:15.945569 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b95c51-318c-4337-bf57-8538916fcde0 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:15.955830 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 778/1585] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:15 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:15.956610 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b95c51-318c-4337-bf57-8538916fcde0 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:15.965994 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b95c51-318c-4337-bf57-8538916fcde0 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:15.971208 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e413b533-4146-4baa-9d51-d93cfadd8ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:15.971844 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e413b533-4146-4baa-9d51-d93cfadd8ae9 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:15.972126 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e413b533-4146-4baa-9d51-d93cfadd8ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:15.972399 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e413b533-4146-4baa-9d51-d93cfadd8ae9 admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:15.973011 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e413b533-4146-4baa-9d51-d93cfadd8ae9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:15.975033 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e413b533-4146-4baa-9d51-d93cfadd8ae9 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:16.009794 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 808/1586] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:20:15 2022] PUT /identity/v3/projects/296b5d138eff48958150d7dda904e484/users/c3b28d23b0f24ec7acc84fb474e5c721/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:16.010661 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37566e6-76a3-42c9-a8b4-b08a703af3a4 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:16.024386 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:16.072249 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37566e6-76a3-42c9-a8b4-b08a703af3a4 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:16.076942 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0028aced-c77b-44d5-aa56-331cba3f4cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:16.077557 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028aced-c77b-44d5-aa56-331cba3f4cc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:16.078375 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028aced-c77b-44d5-aa56-331cba3f4cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:16.078658 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028aced-c77b-44d5-aa56-331cba3f4cc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:16.079304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0028aced-c77b-44d5-aa56-331cba3f4cc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:16.080941 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0028aced-c77b-44d5-aa56-331cba3f4cc8 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:16.087716 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 779/1587] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:20:15 2022] POST /identity/v3/users => generated 287 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:16.087716 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e413b533-4146-4baa-9d51-d93cfadd8ae9 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:16.094131 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 809/1588] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:16 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:16.095645 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e413b533-4146-4baa-9d51-d93cfadd8ae9 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:16.097882 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0028aced-c77b-44d5-aa56-331cba3f4cc8 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:16.100279 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93c24f-fa9e-4bff-8e35-774618672c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:16.100881 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93c24f-fa9e-4bff-8e35-774618672c8c admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:16.111124 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93c24f-fa9e-4bff-8e35-774618672c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:16.111124 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93c24f-fa9e-4bff-8e35-774618672c8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:16.111124 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af93c24f-fa9e-4bff-8e35-774618672c8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:16.111124 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af93c24f-fa9e-4bff-8e35-774618672c8c admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:16.111630 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0028aced-c77b-44d5-aa56-331cba3f4cc8 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:16.113830 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dfb676-35dc-446c-9141-310b991558fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:16.114436 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dfb676-35dc-446c-9141-310b991558fd admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:16.114769 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 780/1589] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:16.115744 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93c24f-fa9e-4bff-8e35-774618672c8c admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:16.115867 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dfb676-35dc-446c-9141-310b991558fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:16.116161 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dfb676-35dc-446c-9141-310b991558fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:16.116963 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dfb676-35dc-446c-9141-310b991558fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:16.118687 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dfb676-35dc-446c-9141-310b991558fd admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:16.125769 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93c24f-fa9e-4bff-8e35-774618672c8c admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:16.129512 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 810/1590] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:16.130675 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48766a7e-3c8e-4fe3-8d78-6b64a2bb4d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:16.130891 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31915bc8-6caf-42d5-bc23-71099977fb67 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:16.131218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31915bc8-6caf-42d5-bc23-71099977fb67 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:16.131392 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48766a7e-3c8e-4fe3-8d78-6b64a2bb4d25 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:16.131584 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48766a7e-3c8e-4fe3-8d78-6b64a2bb4d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:16.131755 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31915bc8-6caf-42d5-bc23-71099977fb67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:16.132018 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48766a7e-3c8e-4fe3-8d78-6b64a2bb4d25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:16.132499 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48766a7e-3c8e-4fe3-8d78-6b64a2bb4d25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:16.134220 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48766a7e-3c8e-4fe3-8d78-6b64a2bb4d25 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:16.146472 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 781/1591] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:16.147589 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48766a7e-3c8e-4fe3-8d78-6b64a2bb4d25 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:16.158112 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48766a7e-3c8e-4fe3-8d78-6b64a2bb4d25 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:16.163303 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d8527-0051-4ccd-91b6-024b44af757c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:16.164227 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d8527-0051-4ccd-91b6-024b44af757c admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:16.164829 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d8527-0051-4ccd-91b6-024b44af757c admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:16.165177 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d8527-0051-4ccd-91b6-024b44af757c admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:16.165893 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d8527-0051-4ccd-91b6-024b44af757c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:16.166018 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-31915bc8-6caf-42d5-bc23-71099977fb67 None None] MFA Rules not processed for user `4cb5b947f84549ecb0145907ecceac63`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:16.167311 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-31915bc8-6caf-42d5-bc23-71099977fb67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:16.167531 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d8527-0051-4ccd-91b6-024b44af757c admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:16.177748 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 782/1592] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:16.178986 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181260d3-aa2a-4262-bc08-407fb7961918 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:16.179278 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181260d3-aa2a-4262-bc08-407fb7961918 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:16.179589 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181260d3-aa2a-4262-bc08-407fb7961918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:16.208182 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-181260d3-aa2a-4262-bc08-407fb7961918 None None] MFA Rules not processed for user `c3b28d23b0f24ec7acc84fb474e5c721`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:16.209441 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-181260d3-aa2a-4262-bc08-407fb7961918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:16.236834 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 811/1593] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:16 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:16.241090 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31915bc8-6caf-42d5-bc23-71099977fb67 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:16.251517 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31915bc8-6caf-42d5-bc23-71099977fb67 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:16.256137 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34172dc8-f531-4d37-855e-50e9875bb920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:16.256724 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34172dc8-f531-4d37-855e-50e9875bb920 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:16.257012 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34172dc8-f531-4d37-855e-50e9875bb920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:16.257293 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34172dc8-f531-4d37-855e-50e9875bb920 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:16.257941 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34172dc8-f531-4d37-855e-50e9875bb920 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:16.259572 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34172dc8-f531-4d37-855e-50e9875bb920 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:16.273882 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 812/1594] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:16.273882 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34172dc8-f531-4d37-855e-50e9875bb920 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:16.278414 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 783/1595] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:16.279533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181260d3-aa2a-4262-bc08-407fb7961918 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:16.282829 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34172dc8-f531-4d37-855e-50e9875bb920 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:16.287522 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2838e24f-4ea7-4657-bdd1-d0572e90cb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:16.287522 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838e24f-4ea7-4657-bdd1-d0572e90cb03 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:16.288553 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838e24f-4ea7-4657-bdd1-d0572e90cb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:16.288678 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838e24f-4ea7-4657-bdd1-d0572e90cb03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:16.289323 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2838e24f-4ea7-4657-bdd1-d0572e90cb03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:16.290957 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2838e24f-4ea7-4657-bdd1-d0572e90cb03 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:16.293855 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181260d3-aa2a-4262-bc08-407fb7961918 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:16.298394 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575d6ab7-9042-4f46-98e3-2734ee082ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:16.299021 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d6ab7-9042-4f46-98e3-2734ee082ad2 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:16.299307 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d6ab7-9042-4f46-98e3-2734ee082ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:16.299629 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d6ab7-9042-4f46-98e3-2734ee082ad2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:16.300286 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575d6ab7-9042-4f46-98e3-2734ee082ad2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:16.301206 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 813/1596] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:16.302032 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575d6ab7-9042-4f46-98e3-2734ee082ad2 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:16.302146 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2838e24f-4ea7-4657-bdd1-d0572e90cb03 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:16.311757 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2838e24f-4ea7-4657-bdd1-d0572e90cb03 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:16.317968 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 784/1597] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:16.317968 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575d6ab7-9042-4f46-98e3-2734ee082ad2 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:16.320043 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40988375-ea4e-4728-8377-ba9a387683ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:16.320870 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40988375-ea4e-4728-8377-ba9a387683ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:16.321161 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40988375-ea4e-4728-8377-ba9a387683ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:16.321444 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40988375-ea4e-4728-8377-ba9a387683ad admin admin] PATH_INFO: `/v3/projects/524406c27cf045a08ac45f2cb420e944/users/26b08395783d4d48916fae7f47ee79eb/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:16.324628 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575d6ab7-9042-4f46-98e3-2734ee082ad2 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:16.330353 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e588d1-6a77-4e65-a05e-d9f25d96dd18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:16.330988 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e588d1-6a77-4e65-a05e-d9f25d96dd18 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:16.331232 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e588d1-6a77-4e65-a05e-d9f25d96dd18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:16.331536 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e588d1-6a77-4e65-a05e-d9f25d96dd18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:16.332175 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e588d1-6a77-4e65-a05e-d9f25d96dd18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:16.333753 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e588d1-6a77-4e65-a05e-d9f25d96dd18 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:16.337751 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40988375-ea4e-4728-8377-ba9a387683ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=524406c27cf045a08ac45f2cb420e944, user_id=26b08395783d4d48916fae7f47ee79eb, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:16.340426 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40988375-ea4e-4728-8377-ba9a387683ad admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:16.344476 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 785/1598] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:16.345449 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e588d1-6a77-4e65-a05e-d9f25d96dd18 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:16.355109 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e588d1-6a77-4e65-a05e-d9f25d96dd18 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:16.359926 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f458d34-9962-43bd-90e5-c842c45cfc80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:16.360557 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f458d34-9962-43bd-90e5-c842c45cfc80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:16.360836 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f458d34-9962-43bd-90e5-c842c45cfc80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:16.361113 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f458d34-9962-43bd-90e5-c842c45cfc80 admin admin] PATH_INFO: `/v3/projects/b1f3e22eb01045f299926f67cd36b91d/users/b5113b1a77fd4b5da175b29b40d2b636/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:16.376256 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f458d34-9962-43bd-90e5-c842c45cfc80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1f3e22eb01045f299926f67cd36b91d, user_id=b5113b1a77fd4b5da175b29b40d2b636, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:16.378931 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f458d34-9962-43bd-90e5-c842c45cfc80 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:16.415527 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 814/1599] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:20:16 2022] PUT /identity/v3/projects/524406c27cf045a08ac45f2cb420e944/users/26b08395783d4d48916fae7f47ee79eb/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:16.416394 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40988375-ea4e-4728-8377-ba9a387683ad admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:16.438152 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 786/1600] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:20:16 2022] PUT /identity/v3/projects/b1f3e22eb01045f299926f67cd36b91d/users/b5113b1a77fd4b5da175b29b40d2b636/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:16.439252 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f458d34-9962-43bd-90e5-c842c45cfc80 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:16.460274 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:16.479643 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40988375-ea4e-4728-8377-ba9a387683ad admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:16.486863 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:16.504250 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f458d34-9962-43bd-90e5-c842c45cfc80 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:16.508978 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211228a-61da-4cfb-9926-56b32d6762de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:16.509593 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211228a-61da-4cfb-9926-56b32d6762de admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:16.509878 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211228a-61da-4cfb-9926-56b32d6762de admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:16.510158 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211228a-61da-4cfb-9926-56b32d6762de admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:16.510809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211228a-61da-4cfb-9926-56b32d6762de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:16.512444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211228a-61da-4cfb-9926-56b32d6762de admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:16.522921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 787/1601] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:16 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:16.523595 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211228a-61da-4cfb-9926-56b32d6762de admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:16.533289 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211228a-61da-4cfb-9926-56b32d6762de admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:16.537802 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b685d31d-3ac1-4b52-900d-2abf4d73c321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:16.538421 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685d31d-3ac1-4b52-900d-2abf4d73c321 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:16.538558 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3ea0d3-c7da-4860-8d33-311aa43cfbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:16.538706 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685d31d-3ac1-4b52-900d-2abf4d73c321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:16.539003 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685d31d-3ac1-4b52-900d-2abf4d73c321 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:16.539107 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ea0d3-c7da-4860-8d33-311aa43cfbdb admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:16.539378 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ea0d3-c7da-4860-8d33-311aa43cfbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:16.543950 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b685d31d-3ac1-4b52-900d-2abf4d73c321 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:16.544067 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ea0d3-c7da-4860-8d33-311aa43cfbdb admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:16.544067 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3ea0d3-c7da-4860-8d33-311aa43cfbdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:16.544250 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b685d31d-3ac1-4b52-900d-2abf4d73c321 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:16.544346 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3ea0d3-c7da-4860-8d33-311aa43cfbdb admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:16.633759 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 815/1602] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:20:16 2022] POST /identity/v3/projects => generated 403 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:16.634718 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b685d31d-3ac1-4b52-900d-2abf4d73c321 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:16.649574 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 788/1603] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:20:16 2022] POST /identity/v3/projects => generated 381 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:16.649574 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3ea0d3-c7da-4860-8d33-311aa43cfbdb admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:16.705172 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b685d31d-3ac1-4b52-900d-2abf4d73c321 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:16.709890 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16408eb4-e132-4e59-9e42-911c5d50a7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:16.710605 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16408eb4-e132-4e59-9e42-911c5d50a7b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:16.710900 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16408eb4-e132-4e59-9e42-911c5d50a7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:16.711181 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16408eb4-e132-4e59-9e42-911c5d50a7b9 admin admin] PATH_INFO: `/v3/projects/965cf5f488364e0ea8b52cdc78de2641/users/2e8380a466484c83bca842518d171048/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:16.712950 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3ea0d3-c7da-4860-8d33-311aa43cfbdb admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:16.717906 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6fad8-2e0d-4af8-b9b1-8d587f51a8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:16.718615 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6fad8-2e0d-4af8-b9b1-8d587f51a8ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:16.718893 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6fad8-2e0d-4af8-b9b1-8d587f51a8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:16.719170 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6fad8-2e0d-4af8-b9b1-8d587f51a8ab admin admin] PATH_INFO: `/v3/projects/a95d3e9a225f45a88b06bee262207b6c/users/a830bd2267f948f693c6b211293c6b89/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:16.726349 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16408eb4-e132-4e59-9e42-911c5d50a7b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=965cf5f488364e0ea8b52cdc78de2641, user_id=2e8380a466484c83bca842518d171048, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:16.729054 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16408eb4-e132-4e59-9e42-911c5d50a7b9 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:16.735056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf6fad8-2e0d-4af8-b9b1-8d587f51a8ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=a95d3e9a225f45a88b06bee262207b6c, user_id=a830bd2267f948f693c6b211293c6b89, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:16.737772 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf6fad8-2e0d-4af8-b9b1-8d587f51a8ab admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:16.780629 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 816/1604] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:20:16 2022] PUT /identity/v3/projects/965cf5f488364e0ea8b52cdc78de2641/users/2e8380a466484c83bca842518d171048/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:16.784429 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16408eb4-e132-4e59-9e42-911c5d50a7b9 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:16.810373 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 789/1605] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:20:16 2022] PUT /identity/v3/projects/a95d3e9a225f45a88b06bee262207b6c/users/a830bd2267f948f693c6b211293c6b89/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:16.811779 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1df200-d49c-44a9-909b-dd9d200e8db5 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:16.815996 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1df200-d49c-44a9-909b-dd9d200e8db5 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:16.815996 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1df200-d49c-44a9-909b-dd9d200e8db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:16.841359 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-ee1df200-d49c-44a9-909b-dd9d200e8db5 None None] MFA Rules not processed for user `26b08395783d4d48916fae7f47ee79eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:16.842599 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-ee1df200-d49c-44a9-909b-dd9d200e8db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:16.842937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:16.848275 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16408eb4-e132-4e59-9e42-911c5d50a7b9 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:16.906687 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89855a0-f83f-46df-9ca0-21c0831a4f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:16.907362 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89855a0-f83f-46df-9ca0-21c0831a4f9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:16.907676 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89855a0-f83f-46df-9ca0-21c0831a4f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:16.908008 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89855a0-f83f-46df-9ca0-21c0831a4f9c admin admin] PATH_INFO: `/v3/projects/74ec48e311334f6cbb2f9bcc1881b378/users/14cf8e880e3247c9a0c36ca7a40894b6/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:16.908113 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 790/1606] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:16.909366 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f177ea-54ca-487a-be0a-bc91d6440bff None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:16.909658 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f177ea-54ca-487a-be0a-bc91d6440bff None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:16.909935 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f177ea-54ca-487a-be0a-bc91d6440bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:16.924647 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89855a0-f83f-46df-9ca0-21c0831a4f9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=74ec48e311334f6cbb2f9bcc1881b378, user_id=14cf8e880e3247c9a0c36ca7a40894b6, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:16.927321 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89855a0-f83f-46df-9ca0-21c0831a4f9c admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:16.929461 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-96f177ea-54ca-487a-be0a-bc91d6440bff None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:16.930653 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-96f177ea-54ca-487a-be0a-bc91d6440bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:16.940154 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 791/1607] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:16.940154 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f177ea-54ca-487a-be0a-bc91d6440bff None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:16.949667 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f177ea-54ca-487a-be0a-bc91d6440bff None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:16.958045 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5eeb0-ad74-4371-b7ad-e6ea153a2be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:16.959046 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5eeb0-ad74-4371-b7ad-e6ea153a2be4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:16.959211 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5eeb0-ad74-4371-b7ad-e6ea153a2be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:16.959522 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5eeb0-ad74-4371-b7ad-e6ea153a2be4 admin admin] PATH_INFO: `/v3/projects/69a97ad9036941329090a7723cac0cb4/users/018b805cff9946c3b5878df51e6332fe/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:16.974492 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e5eeb0-ad74-4371-b7ad-e6ea153a2be4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69a97ad9036941329090a7723cac0cb4, user_id=018b805cff9946c3b5878df51e6332fe, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:16.977170 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e5eeb0-ad74-4371-b7ad-e6ea153a2be4 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:16.981295 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 817/1608] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:20:16 2022] PUT /identity/v3/projects/74ec48e311334f6cbb2f9bcc1881b378/users/14cf8e880e3247c9a0c36ca7a40894b6/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:16.991184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89855a0-f83f-46df-9ca0-21c0831a4f9c admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:17.045597 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89855a0-f83f-46df-9ca0-21c0831a4f9c admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:17.050881 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:17.051738 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f197454-c9e8-4034-bc3f-eeb08c955c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:17.052307 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f197454-c9e8-4034-bc3f-eeb08c955c59 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:17.052595 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f197454-c9e8-4034-bc3f-eeb08c955c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:17.052873 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f197454-c9e8-4034-bc3f-eeb08c955c59 admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:17.053519 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f197454-c9e8-4034-bc3f-eeb08c955c59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:17.055480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f197454-c9e8-4034-bc3f-eeb08c955c59 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:17.056628 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 792/1609] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:20:16 2022] PUT /identity/v3/projects/69a97ad9036941329090a7723cac0cb4/users/018b805cff9946c3b5878df51e6332fe/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:17.057413 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5eeb0-ad74-4371-b7ad-e6ea153a2be4 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:17.138886 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5eeb0-ad74-4371-b7ad-e6ea153a2be4 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:17.144479 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f036d8a-d187-45a4-b441-7fc864b175ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:17.145102 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f036d8a-d187-45a4-b441-7fc864b175ff admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:17.145390 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f036d8a-d187-45a4-b441-7fc864b175ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:17.145671 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f036d8a-d187-45a4-b441-7fc864b175ff admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:17.146317 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f036d8a-d187-45a4-b441-7fc864b175ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:17.148400 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f036d8a-d187-45a4-b441-7fc864b175ff admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:17.164104 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 818/1610] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:20:16 2022] POST /identity/v3/users => generated 308 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:17.164104 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a532f35-b081-4f33-b3e8-12af44ccb3e0 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:17.164104 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a532f35-b081-4f33-b3e8-12af44ccb3e0 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:17.164104 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a532f35-b081-4f33-b3e8-12af44ccb3e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:17.185765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-1a532f35-b081-4f33-b3e8-12af44ccb3e0 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:17.186976 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-1a532f35-b081-4f33-b3e8-12af44ccb3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:17.194757 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 819/1611] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:17.195950 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5968b-7a68-43ef-834d-1188393b7116 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:17.196236 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5968b-7a68-43ef-834d-1188393b7116 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:17.196510 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5968b-7a68-43ef-834d-1188393b7116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:17.219492 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-5ab5968b-7a68-43ef-834d-1188393b7116 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:17.220717 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-5ab5968b-7a68-43ef-834d-1188393b7116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:17.228539 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 820/1612] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:17.229324 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5968b-7a68-43ef-834d-1188393b7116 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:17.247257 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 793/1613] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:20:17 2022] POST /identity/v3/users => generated 297 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:17.247257 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f036d8a-d187-45a4-b441-7fc864b175ff admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:17.286269 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5968b-7a68-43ef-834d-1188393b7116 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:17.290089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640beff-1258-4ecf-92fd-b180ded6eb84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:17.290701 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640beff-1258-4ecf-92fd-b180ded6eb84 service cinder] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:17.290989 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640beff-1258-4ecf-92fd-b180ded6eb84 service cinder] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:17.291269 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640beff-1258-4ecf-92fd-b180ded6eb84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:17.303532 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f036d8a-d187-45a4-b441-7fc864b175ff admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:17.305005 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:17.307337 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf78d15-84f7-4c10-a0ac-2f5e9cbab9d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:17.307974 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf78d15-84f7-4c10-a0ac-2f5e9cbab9d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:17.308257 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf78d15-84f7-4c10-a0ac-2f5e9cbab9d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:17.308532 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf78d15-84f7-4c10-a0ac-2f5e9cbab9d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:17.317978 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf78d15-84f7-4c10-a0ac-2f5e9cbab9d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:17.319444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf78d15-84f7-4c10-a0ac-2f5e9cbab9d0 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:17.329262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 794/1614] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:20:17.330609 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde4ad0-7e5f-49bd-89ae-c789234feddc None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:17.330910 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde4ad0-7e5f-49bd-89ae-c789234feddc None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:17.331193 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde4ad0-7e5f-49bd-89ae-c789234feddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:17.351287 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640beff-1258-4ecf-92fd-b180ded6eb84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:17.352888 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640beff-1258-4ecf-92fd-b180ded6eb84 service cinder] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:17.363025 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 821/1615] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:20:17.363239 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-8bde4ad0-7e5f-49bd-89ae-c789234feddc None None] MFA Rules not processed for user `14cf8e880e3247c9a0c36ca7a40894b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:17.364074 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a9b45-4344-475f-8b13-a3093776d89e None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:17.364367 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a9b45-4344-475f-8b13-a3093776d89e None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:17.364477 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-8bde4ad0-7e5f-49bd-89ae-c789234feddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:17.364649 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a9b45-4344-475f-8b13-a3093776d89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:17.395972 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-2e6a9b45-4344-475f-8b13-a3093776d89e None None] MFA Rules not processed for user `018b805cff9946c3b5878df51e6332fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:17.397255 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-2e6a9b45-4344-475f-8b13-a3093776d89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:17.441894 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 795/1616] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:17.447486 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bde4ad0-7e5f-49bd-89ae-c789234feddc None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:17.456403 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bde4ad0-7e5f-49bd-89ae-c789234feddc None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:17.462291 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6fa23-e6a4-42ad-baa5-ea7c5df48a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:17.462291 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6fa23-e6a4-42ad-baa5-ea7c5df48a63 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:17.462494 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6fa23-e6a4-42ad-baa5-ea7c5df48a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:17.462709 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6fa23-e6a4-42ad-baa5-ea7c5df48a63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:17.463381 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a6fa23-e6a4-42ad-baa5-ea7c5df48a63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:17.465183 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a6fa23-e6a4-42ad-baa5-ea7c5df48a63 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:17.479180 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 796/1617] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:17.480107 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6fa23-e6a4-42ad-baa5-ea7c5df48a63 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:17.482086 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 822/1618] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:17.483520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-83a6fa23-e6a4-42ad-baa5-ea7c5df48a63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:17.483758 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a9b45-4344-475f-8b13-a3093776d89e None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:17.485327 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-2e6a9b45-4344-475f-8b13-a3093776d89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:17.511007 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6fa23-e6a4-42ad-baa5-ea7c5df48a63 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:17.517151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea3f3f-d146-40ec-95ed-2ff4326b197c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:17.518121 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea3f3f-d146-40ec-95ed-2ff4326b197c service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:17.518662 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea3f3f-d146-40ec-95ed-2ff4326b197c service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:17.519193 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea3f3f-d146-40ec-95ed-2ff4326b197c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:17.526873 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a9b45-4344-475f-8b13-a3093776d89e None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:17.530545 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:17.533743 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea3f3f-d146-40ec-95ed-2ff4326b197c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:17.535616 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea3f3f-d146-40ec-95ed-2ff4326b197c service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:17.537616 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f46e3-acc1-43c0-aaac-ab0195f1f360 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:17.538252 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f46e3-acc1-43c0-aaac-ab0195f1f360 service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:17.540238 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f46e3-acc1-43c0-aaac-ab0195f1f360 service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:17.542998 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f46e3-acc1-43c0-aaac-ab0195f1f360 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:17.559856 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 797/1619] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:20:17.560538 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221f46e3-acc1-43c0-aaac-ab0195f1f360 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:17.561432 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea3f3f-d146-40ec-95ed-2ff4326b197c service neutron] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:17.562351 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221f46e3-acc1-43c0-aaac-ab0195f1f360 service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:17.574903 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 823/1620] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:20:17.576340 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f46e3-acc1-43c0-aaac-ab0195f1f360 service neutron] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:17.576462 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea3f3f-d146-40ec-95ed-2ff4326b197c service neutron] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:17.584295 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7cd91e-16eb-4c27-bbaf-8b43e4f04db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:17.585257 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7cd91e-16eb-4c27-bbaf-8b43e4f04db9 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:17.585909 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7cd91e-16eb-4c27-bbaf-8b43e4f04db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:17.586409 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f46e3-acc1-43c0-aaac-ab0195f1f360 service neutron] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:17.586532 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7cd91e-16eb-4c27-bbaf-8b43e4f04db9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:17.587140 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7cd91e-16eb-4c27-bbaf-8b43e4f04db9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:17.589631 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7cd91e-16eb-4c27-bbaf-8b43e4f04db9 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:17.591799 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f3c5c-e4e5-4f2e-98ca-9efe947ce465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:17.592685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f3c5c-e4e5-4f2e-98ca-9efe947ce465 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:17.592832 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f3c5c-e4e5-4f2e-98ca-9efe947ce465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:17.592995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f3c5c-e4e5-4f2e-98ca-9efe947ce465 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:17.594252 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795f3c5c-e4e5-4f2e-98ca-9efe947ce465 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:17.596368 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795f3c5c-e4e5-4f2e-98ca-9efe947ce465 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:17.603671 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 798/1621] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:17.604586 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7cd91e-16eb-4c27-bbaf-8b43e4f04db9 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:17.617237 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7cd91e-16eb-4c27-bbaf-8b43e4f04db9 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:17.622610 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d012ac43-ed5c-4a8a-8a34-6e28bcc827b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:17.623578 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012ac43-ed5c-4a8a-8a34-6e28bcc827b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:17.623861 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012ac43-ed5c-4a8a-8a34-6e28bcc827b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:17.624142 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012ac43-ed5c-4a8a-8a34-6e28bcc827b4 admin admin] PATH_INFO: `/v3/projects/eaa66bc30f22452e855c8d5a8a53e525/users/fac4668c920c4fb6a5b8e5a386bfc9e3/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:17.643957 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d012ac43-ed5c-4a8a-8a34-6e28bcc827b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaa66bc30f22452e855c8d5a8a53e525, user_id=fac4668c920c4fb6a5b8e5a386bfc9e3, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:17.647080 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d012ac43-ed5c-4a8a-8a34-6e28bcc827b4 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:17.720738 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 824/1622] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:20:17 2022] POST /identity/v3/projects => generated 373 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:17.723955 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36f5dd-3aa1-4784-b1e4-f4b8c8f42c33 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:17.723955 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36f5dd-3aa1-4784-b1e4-f4b8c8f42c33 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:17.723955 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36f5dd-3aa1-4784-b1e4-f4b8c8f42c33 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:17.724971 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 825/1623] 10.222.0.98 () {54 vars in 962 bytes} [Thu Aug 18 12:20:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:20:17.725803 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d36f5dd-3aa1-4784-b1e4-f4b8c8f42c33 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:17.750153 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 799/1624] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:20:17 2022] PUT /identity/v3/projects/eaa66bc30f22452e855c8d5a8a53e525/users/fac4668c920c4fb6a5b8e5a386bfc9e3/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:17.751717 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d012ac43-ed5c-4a8a-8a34-6e28bcc827b4 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:17.837032 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d012ac43-ed5c-4a8a-8a34-6e28bcc827b4 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:17.837618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d36f5dd-3aa1-4784-b1e4-f4b8c8f42c33 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:17.842334 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e6a0a-f4a3-4613-8e98-921bda722832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:17.843062 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e6a0a-f4a3-4613-8e98-921bda722832 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:17.843215 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812b288-1995-4a79-aaf0-a3c3df1a9329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:17.843507 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e6a0a-f4a3-4613-8e98-921bda722832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:17.844012 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e6a0a-f4a3-4613-8e98-921bda722832 admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:17.844119 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812b288-1995-4a79-aaf0-a3c3df1a9329 admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:17.844378 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812b288-1995-4a79-aaf0-a3c3df1a9329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:17.844662 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812b288-1995-4a79-aaf0-a3c3df1a9329 admin admin] PATH_INFO: `/v3/projects/0a3aed9d3ac44f0287f88dc207cd9c14/users/3e5b3f0e256a428cba079f95d2c9a0ee/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:17.844781 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e6a0a-f4a3-4613-8e98-921bda722832 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:17.846792 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e6a0a-f4a3-4613-8e98-921bda722832 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:17.861227 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812b288-1995-4a79-aaf0-a3c3df1a9329 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a3aed9d3ac44f0287f88dc207cd9c14, user_id=3e5b3f0e256a428cba079f95d2c9a0ee, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:17.863986 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812b288-1995-4a79-aaf0-a3c3df1a9329 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:17.866240 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:17.945698 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 826/1625] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:20:17 2022] PUT /identity/v3/projects/0a3aed9d3ac44f0287f88dc207cd9c14/users/3e5b3f0e256a428cba079f95d2c9a0ee/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:17.946626 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 800/1626] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:20:17 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:17.947277 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990ff88-43b2-4b89-bf81-abcac596867c None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:17.947601 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990ff88-43b2-4b89-bf81-abcac596867c None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:17.947880 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990ff88-43b2-4b89-bf81-abcac596867c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:17.948115 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e6a0a-f4a3-4613-8e98-921bda722832 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:17.963806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e6a0a-f4a3-4613-8e98-921bda722832 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:18.008614 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-a990ff88-43b2-4b89-bf81-abcac596867c None None] MFA Rules not processed for user `171c72a6b29e418bbb4ea5c6faffe3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:18.010013 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-a990ff88-43b2-4b89-bf81-abcac596867c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:18.064937 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0599c3d-a545-47fc-b8d5-f12e1ef01604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:18.065682 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0599c3d-a545-47fc-b8d5-f12e1ef01604 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:18.066061 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0599c3d-a545-47fc-b8d5-f12e1ef01604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:18.066405 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0599c3d-a545-47fc-b8d5-f12e1ef01604 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:18.067134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0599c3d-a545-47fc-b8d5-f12e1ef01604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:18.068989 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0599c3d-a545-47fc-b8d5-f12e1ef01604 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:18.080586 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 801/1627] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:18.081656 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0599c3d-a545-47fc-b8d5-f12e1ef01604 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:18.116179 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0599c3d-a545-47fc-b8d5-f12e1ef01604 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:18.118024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 827/1628] 10.222.0.98 () {58 vars in 1059 bytes} [Thu Aug 18 12:20:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:18.118913 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a990ff88-43b2-4b89-bf81-abcac596867c None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:18.122053 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e4d1a-a33c-4d47-9a5f-d7e0b2c79304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:18.123113 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e4d1a-a33c-4d47-9a5f-d7e0b2c79304 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:18.123578 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e4d1a-a33c-4d47-9a5f-d7e0b2c79304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:18.124066 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e4d1a-a33c-4d47-9a5f-d7e0b2c79304 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:18.125459 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422e4d1a-a33c-4d47-9a5f-d7e0b2c79304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:18.128231 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422e4d1a-a33c-4d47-9a5f-d7e0b2c79304 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:18.130697 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a990ff88-43b2-4b89-bf81-abcac596867c None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:18.137044 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813a379a-6c11-4531-94eb-c444d29bda42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:18.139383 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a379a-6c11-4531-94eb-c444d29bda42 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:18.139383 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a379a-6c11-4531-94eb-c444d29bda42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:18.139383 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a379a-6c11-4531-94eb-c444d29bda42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:18.139383 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813a379a-6c11-4531-94eb-c444d29bda42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:18.141035 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813a379a-6c11-4531-94eb-c444d29bda42 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:18.147873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 802/1629] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:18.149097 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e4d1a-a33c-4d47-9a5f-d7e0b2c79304 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:18.157449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 828/1630] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:18.158355 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813a379a-6c11-4531-94eb-c444d29bda42 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:18.165401 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e4d1a-a33c-4d47-9a5f-d7e0b2c79304 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:18.173421 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68496c38-6f17-43f3-bbcf-313b0160c8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:18.174121 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68496c38-6f17-43f3-bbcf-313b0160c8bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:18.174465 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68496c38-6f17-43f3-bbcf-313b0160c8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:18.174901 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68496c38-6f17-43f3-bbcf-313b0160c8bb admin admin] PATH_INFO: `/v3/projects/eaa66bc30f22452e855c8d5a8a53e525/users/fac4668c920c4fb6a5b8e5a386bfc9e3/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:18.176134 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813a379a-6c11-4531-94eb-c444d29bda42 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:18.177122 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68496c38-6f17-43f3-bbcf-313b0160c8bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaa66bc30f22452e855c8d5a8a53e525, user_id=fac4668c920c4fb6a5b8e5a386bfc9e3, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:18.181879 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68496c38-6f17-43f3-bbcf-313b0160c8bb admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:18.183758 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ab7ff-ca18-45b2-8cc6-9252c5b7a807 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:18.184457 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ab7ff-ca18-45b2-8cc6-9252c5b7a807 service nova] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:18.184790 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ab7ff-ca18-45b2-8cc6-9252c5b7a807 service nova] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:18.185106 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ab7ff-ca18-45b2-8cc6-9252c5b7a807 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:18.217861 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 803/1631] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:20:18 2022] PUT /identity/v3/projects/eaa66bc30f22452e855c8d5a8a53e525/users/fac4668c920c4fb6a5b8e5a386bfc9e3/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:18.231389 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68496c38-6f17-43f3-bbcf-313b0160c8bb admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:18.265029 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:18.298978 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740ab7ff-ca18-45b2-8cc6-9252c5b7a807 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:18.300688 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740ab7ff-ca18-45b2-8cc6-9252c5b7a807 service nova] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:18.312484 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 829/1632] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:20:18.322654 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68496c38-6f17-43f3-bbcf-313b0160c8bb admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:18.322654 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c227d05-284d-4df9-8e92-022f961b8e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:18.323059 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c227d05-284d-4df9-8e92-022f961b8e55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:18.323059 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c227d05-284d-4df9-8e92-022f961b8e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:18.323397 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c227d05-284d-4df9-8e92-022f961b8e55 admin admin] PATH_INFO: `/v3/projects/0a3aed9d3ac44f0287f88dc207cd9c14/users/3e5b3f0e256a428cba079f95d2c9a0ee/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:18.323502 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ab7ff-ca18-45b2-8cc6-9252c5b7a807 service nova] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:18.325464 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c227d05-284d-4df9-8e92-022f961b8e55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a3aed9d3ac44f0287f88dc207cd9c14, user_id=3e5b3f0e256a428cba079f95d2c9a0ee, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:18.328982 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c227d05-284d-4df9-8e92-022f961b8e55 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:18.341272 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 804/1633] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:20:18 2022] PUT /identity/v3/projects/0a3aed9d3ac44f0287f88dc207cd9c14/users/3e5b3f0e256a428cba079f95d2c9a0ee/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:18.342682 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d66e1-92b9-482b-8dd3-199af28948b2 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:18.342972 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d66e1-92b9-482b-8dd3-199af28948b2 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:18.343250 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d66e1-92b9-482b-8dd3-199af28948b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:18.359498 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ab7ff-ca18-45b2-8cc6-9252c5b7a807 service nova] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:18.373982 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-9f2d66e1-92b9-482b-8dd3-199af28948b2 None None] MFA Rules not processed for user `fac4668c920c4fb6a5b8e5a386bfc9e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:18.375266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-9f2d66e1-92b9-482b-8dd3-199af28948b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:18.458905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23216942-fbd2-40a8-ba57-4c590a031588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:18.459628 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23216942-fbd2-40a8-ba57-4c590a031588 admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:18.460330 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23216942-fbd2-40a8-ba57-4c590a031588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:18.469286 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23216942-fbd2-40a8-ba57-4c590a031588 admin admin] PATH_INFO: `/v3/projects/bdf7b54a8ed14ca7b73c7972b039309f/users/8134ec0bede6493e8c8e8e9ce2a5b800/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:18.479617 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:18.525815 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 805/1634] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:18 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:18.527229 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da8b7f-6af3-47e8-ac2a-f73c95b4af75 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:18.527543 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da8b7f-6af3-47e8-ac2a-f73c95b4af75 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:18.527818 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da8b7f-6af3-47e8-ac2a-f73c95b4af75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:18.540918 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23216942-fbd2-40a8-ba57-4c590a031588 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdf7b54a8ed14ca7b73c7972b039309f, user_id=8134ec0bede6493e8c8e8e9ce2a5b800, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:18.547483 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23216942-fbd2-40a8-ba57-4c590a031588 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:18.594119 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-94da8b7f-6af3-47e8-ac2a-f73c95b4af75 None None] MFA Rules not processed for user `3e5b3f0e256a428cba079f95d2c9a0ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:18.595383 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-94da8b7f-6af3-47e8-ac2a-f73c95b4af75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:18.656303 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 830/1635] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:20:18 2022] PUT /identity/v3/projects/bdf7b54a8ed14ca7b73c7972b039309f/users/8134ec0bede6493e8c8e8e9ce2a5b800/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:18.657005 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23216942-fbd2-40a8-ba57-4c590a031588 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:18.760966 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23216942-fbd2-40a8-ba57-4c590a031588 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:18.761217 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 806/1636] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:18 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:18.766462 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da8b7f-6af3-47e8-ac2a-f73c95b4af75 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:18.771341 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a04587-4036-4457-9686-8d665d3a4bb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:18.772296 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a04587-4036-4457-9686-8d665d3a4bb7 service cinder] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:18.773042 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a04587-4036-4457-9686-8d665d3a4bb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:18.773578 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a04587-4036-4457-9686-8d665d3a4bb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:18.779207 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da8b7f-6af3-47e8-ac2a-f73c95b4af75 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:18.809622 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:18.861175 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5500e2f-3f93-4c9b-b35b-4019c1112e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:18.862033 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5500e2f-3f93-4c9b-b35b-4019c1112e38 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:18.862563 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5500e2f-3f93-4c9b-b35b-4019c1112e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:18.862840 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5500e2f-3f93-4c9b-b35b-4019c1112e38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:18.864252 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5500e2f-3f93-4c9b-b35b-4019c1112e38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:18.866134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5500e2f-3f93-4c9b-b35b-4019c1112e38 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:18.866644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a04587-4036-4457-9686-8d665d3a4bb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:18.868279 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a04587-4036-4457-9686-8d665d3a4bb7 service cinder] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:18.881403 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 807/1637] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:18.881403 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2bba02-8c5a-49ee-bae8-a4a90d7b2dae None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:18.881403 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2bba02-8c5a-49ee-bae8-a4a90d7b2dae None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:18.881403 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2bba02-8c5a-49ee-bae8-a4a90d7b2dae None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:18.881403 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 808/1638] 10.222.0.98 () {54 vars in 912 bytes} [Thu Aug 18 12:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:20:18.882485 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2bba02-8c5a-49ee-bae8-a4a90d7b2dae None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:18.887617 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 831/1639] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:18 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:20:18.888901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a04587-4036-4457-9686-8d665d3a4bb7 service cinder] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:18.894502 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2bba02-8c5a-49ee-bae8-a4a90d7b2dae None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:18.899007 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a7c43-5c97-4929-a396-c5e33476fdfc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:18.899717 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a7c43-5c97-4929-a396-c5e33476fdfc service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:18.900093 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a7c43-5c97-4929-a396-c5e33476fdfc service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:18.900444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a7c43-5c97-4929-a396-c5e33476fdfc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:18.901753 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a04587-4036-4457-9686-8d665d3a4bb7 service cinder] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:18.907724 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a11bd02-16a2-496b-83be-4e37a0cfb2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:18.909297 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11bd02-16a2-496b-83be-4e37a0cfb2e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:18.909297 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11bd02-16a2-496b-83be-4e37a0cfb2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:18.909297 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11bd02-16a2-496b-83be-4e37a0cfb2e6 admin admin] PATH_INFO: `/v3/projects/bdf7b54a8ed14ca7b73c7972b039309f/users/8134ec0bede6493e8c8e8e9ce2a5b800/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:18.911147 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a11bd02-16a2-496b-83be-4e37a0cfb2e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdf7b54a8ed14ca7b73c7972b039309f, user_id=8134ec0bede6493e8c8e8e9ce2a5b800, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:18.914201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a11bd02-16a2-496b-83be-4e37a0cfb2e6 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:18.926937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 832/1640] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:20:18 2022] PUT /identity/v3/projects/bdf7b54a8ed14ca7b73c7972b039309f/users/8134ec0bede6493e8c8e8e9ce2a5b800/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:18.931770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884d246-4ecf-49bc-964b-0ca1e257b322 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:18.931770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884d246-4ecf-49bc-964b-0ca1e257b322 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:18.931770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884d246-4ecf-49bc-964b-0ca1e257b322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:18.964495 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-4884d246-4ecf-49bc-964b-0ca1e257b322 None None] MFA Rules not processed for user `171c72a6b29e418bbb4ea5c6faffe3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:18.965247 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-4884d246-4ecf-49bc-964b-0ca1e257b322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:19.011743 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a7c43-5c97-4929-a396-c5e33476fdfc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:19.022310 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a7c43-5c97-4929-a396-c5e33476fdfc service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:19.039926 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 809/1641] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:18 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:20:19.043537 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9591d0-8e20-497d-b5f4-7016dde92aaa None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:19.043537 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9591d0-8e20-497d-b5f4-7016dde92aaa None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:19.043537 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9591d0-8e20-497d-b5f4-7016dde92aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:19.096090 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-ae9591d0-8e20-497d-b5f4-7016dde92aaa None None] MFA Rules not processed for user `8134ec0bede6493e8c8e8e9ce2a5b800`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:19.097987 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-ae9591d0-8e20-497d-b5f4-7016dde92aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:19.146073 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 833/1642] 10.222.0.98 () {58 vars in 1009 bytes} [Thu Aug 18 12:20:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:19.147181 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884d246-4ecf-49bc-964b-0ca1e257b322 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:19.172788 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:19.230104 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 810/1643] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:19 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:19.232060 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9591d0-8e20-497d-b5f4-7016dde92aaa None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:19.256763 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884d246-4ecf-49bc-964b-0ca1e257b322 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:19.263477 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ddb23-9a51-406c-91a6-eceb8f5901cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:19.265776 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ddb23-9a51-406c-91a6-eceb8f5901cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:19.265776 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ddb23-9a51-406c-91a6-eceb8f5901cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:19.266963 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ddb23-9a51-406c-91a6-eceb8f5901cd admin admin] PATH_INFO: `/v3/users/4cb5b947f84549ecb0145907ecceac63` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:19.268571 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ddb23-9a51-406c-91a6-eceb8f5901cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cb5b947f84549ecb0145907ecceac63)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:19.269864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: WARNING py.warnings [None req-108ddb23-9a51-406c-91a6-eceb8f5901cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 12:20:19.269864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: warnings.warn(msg) Aug 18 12:20:19.269864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: Aug 18 12:20:19.272167 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ddb23-9a51-406c-91a6-eceb8f5901cd admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:19.307972 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:19.336044 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9591d0-8e20-497d-b5f4-7016dde92aaa None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:19.340878 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ccc217-ca84-4dc8-9d50-e8367b36ae37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:19.341641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ccc217-ca84-4dc8-9d50-e8367b36ae37 service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:19.341980 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ccc217-ca84-4dc8-9d50-e8367b36ae37 service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:19.342257 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ccc217-ca84-4dc8-9d50-e8367b36ae37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:19.357609 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ccc217-ca84-4dc8-9d50-e8367b36ae37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:19.359135 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ccc217-ca84-4dc8-9d50-e8367b36ae37 service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:19.371241 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 811/1644] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:20:19.372116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ccc217-ca84-4dc8-9d50-e8367b36ae37 service cinder] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:19.381551 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ccc217-ca84-4dc8-9d50-e8367b36ae37 service cinder] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:19.385814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3cb947-c952-4bf4-9dfe-32178cc5d363 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:19.386481 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cb947-c952-4bf4-9dfe-32178cc5d363 service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:19.386831 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cb947-c952-4bf4-9dfe-32178cc5d363 service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:19.387182 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cb947-c952-4bf4-9dfe-32178cc5d363 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:19.401362 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3cb947-c952-4bf4-9dfe-32178cc5d363 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:19.402741 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3cb947-c952-4bf4-9dfe-32178cc5d363 service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:19.413827 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 812/1645] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:19 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:20:19.579715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-108ddb23-9a51-406c-91a6-eceb8f5901cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cb5b947f84549ecb0145907ecceac63'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:19.594173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-108ddb23-9a51-406c-91a6-eceb8f5901cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cb5b947f84549ecb0145907ecceac63'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:19.595992 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-108ddb23-9a51-406c-91a6-eceb8f5901cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cb5b947f84549ecb0145907ecceac63'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:19.619489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-108ddb23-9a51-406c-91a6-eceb8f5901cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cb5b947f84549ecb0145907ecceac63'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:19.650000 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3cb947-c952-4bf4-9dfe-32178cc5d363 service cinder] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:19.658628 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3cb947-c952-4bf4-9dfe-32178cc5d363 service cinder] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:19.662320 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f1340-c998-4d14-9711-b3c80a3a39f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:19.662925 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f1340-c998-4d14-9711-b3c80a3a39f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:19.663209 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f1340-c998-4d14-9711-b3c80a3a39f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:19.663565 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f1340-c998-4d14-9711-b3c80a3a39f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:19.724125 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-108ddb23-9a51-406c-91a6-eceb8f5901cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cb5b947f84549ecb0145907ecceac63'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:19.731260 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 834/1646] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:20:19 2022] DELETE /identity/v3/users/4cb5b947f84549ecb0145907ecceac63 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:19.740720 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8b512-76cb-4b28-8852-ec67a195a1c1 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:19.741014 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8b512-76cb-4b28-8852-ec67a195a1c1 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:19.741302 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8b512-76cb-4b28-8852-ec67a195a1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:19.763511 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-62d8b512-76cb-4b28-8852-ec67a195a1c1 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:19.764766 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-62d8b512-76cb-4b28-8852-ec67a195a1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:19.768554 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f1340-c998-4d14-9711-b3c80a3a39f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:19.770026 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f1340-c998-4d14-9711-b3c80a3a39f8 service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:19.786474 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 813/1647] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:20:19.867321 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 835/1648] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:20:19.893037 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f1340-c998-4d14-9711-b3c80a3a39f8 service cinder] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:19.959936 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f1340-c998-4d14-9711-b3c80a3a39f8 service cinder] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:20.064250 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e5ede-09bd-4bea-b5ca-75bec8f5e875 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:20.065502 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e5ede-09bd-4bea-b5ca-75bec8f5e875 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:20.065992 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e5ede-09bd-4bea-b5ca-75bec8f5e875 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:20.066371 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e5ede-09bd-4bea-b5ca-75bec8f5e875 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:20.079315 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73e5ede-09bd-4bea-b5ca-75bec8f5e875 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:20.081064 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73e5ede-09bd-4bea-b5ca-75bec8f5e875 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:20.095542 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 814/1649] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:20:20.511984 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e99cba-7e21-43e0-af03-502821d9c1ac None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:20.511984 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e99cba-7e21-43e0-af03-502821d9c1ac None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:20.511984 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e99cba-7e21-43e0-af03-502821d9c1ac None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:20.511984 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 836/1650] 10.222.0.98 () {54 vars in 962 bytes} [Thu Aug 18 12:20:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:20:20.544525 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36983419-1dde-4698-b8fc-d40d1687942f None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:20.545164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36983419-1dde-4698-b8fc-d40d1687942f None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:20.545164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36983419-1dde-4698-b8fc-d40d1687942f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:20.587039 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-36983419-1dde-4698-b8fc-d40d1687942f None None] MFA Rules not processed for user `171c72a6b29e418bbb4ea5c6faffe3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:20.588584 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-36983419-1dde-4698-b8fc-d40d1687942f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:20.639286 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:20.698951 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 815/1651] 10.222.0.98 () {58 vars in 1059 bytes} [Thu Aug 18 12:20:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:20.704809 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa3aad-54f6-40b1-9ec1-81ad2db4afa7 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:20.705093 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa3aad-54f6-40b1-9ec1-81ad2db4afa7 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:20.705369 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa3aad-54f6-40b1-9ec1-81ad2db4afa7 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:20.706718 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 837/1652] 10.222.0.98 () {54 vars in 912 bytes} [Thu Aug 18 12:20:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:20:20.713570 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27abe23-c1cd-4152-8ed1-118f3472cd03 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:20.713924 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27abe23-c1cd-4152-8ed1-118f3472cd03 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:20.718839 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27abe23-c1cd-4152-8ed1-118f3472cd03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:20.767375 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-f27abe23-c1cd-4152-8ed1-118f3472cd03 None None] MFA Rules not processed for user `171c72a6b29e418bbb4ea5c6faffe3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:20.770081 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-f27abe23-c1cd-4152-8ed1-118f3472cd03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:20.785971 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 816/1653] 10.222.0.98 () {58 vars in 1009 bytes} [Thu Aug 18 12:20:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:20.891312 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fa3aad-54f6-40b1-9ec1-81ad2db4afa7 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:20.894261 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-87fa3aad-54f6-40b1-9ec1-81ad2db4afa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:20.957168 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:20.990142 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fa3aad-54f6-40b1-9ec1-81ad2db4afa7 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:20.996199 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3fed6-6d0f-4949-8b7e-7efdfbb7c18d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:20.997456 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3fed6-6d0f-4949-8b7e-7efdfbb7c18d service cinder] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:20.999239 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3fed6-6d0f-4949-8b7e-7efdfbb7c18d service cinder] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:21.000467 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3fed6-6d0f-4949-8b7e-7efdfbb7c18d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:21.016306 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3fed6-6d0f-4949-8b7e-7efdfbb7c18d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:21.019588 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3fed6-6d0f-4949-8b7e-7efdfbb7c18d service cinder] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:21.039689 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 838/1654] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:20:21.545598 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27abe23-c1cd-4152-8ed1-118f3472cd03 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:21.548461 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-f27abe23-c1cd-4152-8ed1-118f3472cd03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:21.594891 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27abe23-c1cd-4152-8ed1-118f3472cd03 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:21.603275 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a715336-c5b9-4138-83d7-c60e36382400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:21.604674 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a715336-c5b9-4138-83d7-c60e36382400 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:21.605167 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a715336-c5b9-4138-83d7-c60e36382400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:21.605649 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a715336-c5b9-4138-83d7-c60e36382400 admin admin] PATH_INFO: `/v3/projects/edb4c111c6354e17a9ff18ac60aba626` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:21.607399 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a715336-c5b9-4138-83d7-c60e36382400 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edb4c111c6354e17a9ff18ac60aba626)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:21.609515 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: WARNING py.warnings [None req-0a715336-c5b9-4138-83d7-c60e36382400 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 18 12:20:21.609515 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: warnings.warn(msg) Aug 18 12:20:21.609515 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: Aug 18 12:20:21.611763 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a715336-c5b9-4138-83d7-c60e36382400 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:21.632627 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3fed6-6d0f-4949-8b7e-7efdfbb7c18d service cinder] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:21.635688 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-b0c3fed6-6d0f-4949-8b7e-7efdfbb7c18d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:21.679182 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3fed6-6d0f-4949-8b7e-7efdfbb7c18d service cinder] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:21.689081 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a258ca3-aba8-47b5-80bb-bf809f1642fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:21.693683 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a258ca3-aba8-47b5-80bb-bf809f1642fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:21.693866 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a258ca3-aba8-47b5-80bb-bf809f1642fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:21.694160 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a258ca3-aba8-47b5-80bb-bf809f1642fa admin admin] PATH_INFO: `/v3/users/c3b28d23b0f24ec7acc84fb474e5c721` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:21.695563 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a258ca3-aba8-47b5-80bb-bf809f1642fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3b28d23b0f24ec7acc84fb474e5c721)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:21.700137 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a258ca3-aba8-47b5-80bb-bf809f1642fa admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:21.863183 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-0a715336-c5b9-4138-83d7-c60e36382400 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edb4c111c6354e17a9ff18ac60aba626', 'request_id': 'req-0a715336-c5b9-4138-83d7-c60e36382400'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:21.871211 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-0a715336-c5b9-4138-83d7-c60e36382400 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edb4c111c6354e17a9ff18ac60aba626', 'request_id': 'req-0a715336-c5b9-4138-83d7-c60e36382400'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:21.958180 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-0a715336-c5b9-4138-83d7-c60e36382400 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edb4c111c6354e17a9ff18ac60aba626', 'request_id': 'req-0a715336-c5b9-4138-83d7-c60e36382400'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:21.983023 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-0a715336-c5b9-4138-83d7-c60e36382400 admin admin] The token cache is being invalidate because project edb4c111c6354e17a9ff18ac60aba626 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:20:21.983640 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-0a715336-c5b9-4138-83d7-c60e36382400 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:21.984603 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-0a715336-c5b9-4138-83d7-c60e36382400 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:21.986614 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 817/1655] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:20:21 2022] DELETE /identity/v3/projects/edb4c111c6354e17a9ff18ac60aba626 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:21.991601 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a715336-c5b9-4138-83d7-c60e36382400 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:21.995537 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-0a715336-c5b9-4138-83d7-c60e36382400 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:22.015693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-7a258ca3-aba8-47b5-80bb-bf809f1642fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3b28d23b0f24ec7acc84fb474e5c721'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:22.036132 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-7a258ca3-aba8-47b5-80bb-bf809f1642fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3b28d23b0f24ec7acc84fb474e5c721'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:22.037140 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-7a258ca3-aba8-47b5-80bb-bf809f1642fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3b28d23b0f24ec7acc84fb474e5c721'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:22.111280 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-7a258ca3-aba8-47b5-80bb-bf809f1642fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3b28d23b0f24ec7acc84fb474e5c721'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:22.160825 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a715336-c5b9-4138-83d7-c60e36382400 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:22.175967 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95c336-4650-4d4a-a968-d77c88086d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:22.176632 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95c336-4650-4d4a-a968-d77c88086d1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:22.176912 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95c336-4650-4d4a-a968-d77c88086d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:22.177185 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95c336-4650-4d4a-a968-d77c88086d1e admin admin] PATH_INFO: `/v3/users/fac4668c920c4fb6a5b8e5a386bfc9e3` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:22.179272 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95c336-4650-4d4a-a968-d77c88086d1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fac4668c920c4fb6a5b8e5a386bfc9e3)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:22.181759 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95c336-4650-4d4a-a968-d77c88086d1e admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:22.209718 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-7a258ca3-aba8-47b5-80bb-bf809f1642fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3b28d23b0f24ec7acc84fb474e5c721'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:22.221457 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 839/1656] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:20:21 2022] DELETE /identity/v3/users/c3b28d23b0f24ec7acc84fb474e5c721 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:20:22.222684 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a258ca3-aba8-47b5-80bb-bf809f1642fa admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:22.233323 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-7a258ca3-aba8-47b5-80bb-bf809f1642fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:22.356769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a258ca3-aba8-47b5-80bb-bf809f1642fa admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:22.361707 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226864f9-4afe-43d9-a894-0c0fb10fbb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:22.362378 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226864f9-4afe-43d9-a894-0c0fb10fbb85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:22.363342 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226864f9-4afe-43d9-a894-0c0fb10fbb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:22.363342 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226864f9-4afe-43d9-a894-0c0fb10fbb85 admin admin] PATH_INFO: `/v3/users/14cf8e880e3247c9a0c36ca7a40894b6` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:22.364223 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226864f9-4afe-43d9-a894-0c0fb10fbb85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14cf8e880e3247c9a0c36ca7a40894b6)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:22.366194 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226864f9-4afe-43d9-a894-0c0fb10fbb85 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:22.947946 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-226864f9-4afe-43d9-a894-0c0fb10fbb85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14cf8e880e3247c9a0c36ca7a40894b6'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:22.958644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-226864f9-4afe-43d9-a894-0c0fb10fbb85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14cf8e880e3247c9a0c36ca7a40894b6'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:22.967360 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-226864f9-4afe-43d9-a894-0c0fb10fbb85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14cf8e880e3247c9a0c36ca7a40894b6'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:22.974172 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:22.979052 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-226864f9-4afe-43d9-a894-0c0fb10fbb85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14cf8e880e3247c9a0c36ca7a40894b6'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:23.002868 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-dc95c336-4650-4d4a-a968-d77c88086d1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fac4668c920c4fb6a5b8e5a386bfc9e3'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:23.003607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-dc95c336-4650-4d4a-a968-d77c88086d1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fac4668c920c4fb6a5b8e5a386bfc9e3'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:23.007999 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-dc95c336-4650-4d4a-a968-d77c88086d1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fac4668c920c4fb6a5b8e5a386bfc9e3'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:23.026329 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-dc95c336-4650-4d4a-a968-d77c88086d1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fac4668c920c4fb6a5b8e5a386bfc9e3'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:23.055136 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-dc95c336-4650-4d4a-a968-d77c88086d1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fac4668c920c4fb6a5b8e5a386bfc9e3'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:23.100390 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-226864f9-4afe-43d9-a894-0c0fb10fbb85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14cf8e880e3247c9a0c36ca7a40894b6'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:23.116237 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 840/1657] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:20:22 2022] DELETE /identity/v3/users/14cf8e880e3247c9a0c36ca7a40894b6 => generated 0 bytes in 898 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:23.120975 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68fe4a-4d45-4c5d-a0c3-f11d4aac7c7e None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:23.121269 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68fe4a-4d45-4c5d-a0c3-f11d4aac7c7e None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:23.121561 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68fe4a-4d45-4c5d-a0c3-f11d4aac7c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:23.199788 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 818/1658] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:20:21 2022] DELETE /identity/v3/users/fac4668c920c4fb6a5b8e5a386bfc9e3 => generated 0 bytes in 1208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:23.204173 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4103a-085c-416a-b9ab-cef8424522e0 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:23.204173 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4103a-085c-416a-b9ab-cef8424522e0 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:23.204173 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4103a-085c-416a-b9ab-cef8424522e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:23.206645 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-ff68fe4a-4d45-4c5d-a0c3-f11d4aac7c7e None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:23.214912 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-ff68fe4a-4d45-4c5d-a0c3-f11d4aac7c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:23.277220 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-37b4103a-085c-416a-b9ab-cef8424522e0 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:23.281542 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-37b4103a-085c-416a-b9ab-cef8424522e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:23.399290 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 841/1659] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:23.409625 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68fe4a-4d45-4c5d-a0c3-f11d4aac7c7e None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:23.411363 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-ff68fe4a-4d45-4c5d-a0c3-f11d4aac7c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:23.439904 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 819/1660] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:23.540245 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b4103a-085c-416a-b9ab-cef8424522e0 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:23.544758 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-37b4103a-085c-416a-b9ab-cef8424522e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:23.591777 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68fe4a-4d45-4c5d-a0c3-f11d4aac7c7e None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:23.604523 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9480202a-3eb5-4009-91d6-0a1ba10bd750 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:23.605173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480202a-3eb5-4009-91d6-0a1ba10bd750 service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:23.605460 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480202a-3eb5-4009-91d6-0a1ba10bd750 service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:23.605738 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480202a-3eb5-4009-91d6-0a1ba10bd750 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:23.629540 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9480202a-3eb5-4009-91d6-0a1ba10bd750 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:23.631392 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9480202a-3eb5-4009-91d6-0a1ba10bd750 service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:23.663548 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 842/1661] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:20:23.751131 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b4103a-085c-416a-b9ab-cef8424522e0 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:23.751131 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa07b6a-0540-4e91-b4bb-f1eddfd26f86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:23.751131 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa07b6a-0540-4e91-b4bb-f1eddfd26f86 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:23.751131 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa07b6a-0540-4e91-b4bb-f1eddfd26f86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:23.752634 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa07b6a-0540-4e91-b4bb-f1eddfd26f86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:23.754609 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa07b6a-0540-4e91-b4bb-f1eddfd26f86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:23.756332 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa07b6a-0540-4e91-b4bb-f1eddfd26f86 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:23.786805 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 820/1662] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:20:24.217085 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9480202a-3eb5-4009-91d6-0a1ba10bd750 service neutron] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:24.218596 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-9480202a-3eb5-4009-91d6-0a1ba10bd750 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:24.246478 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9480202a-3eb5-4009-91d6-0a1ba10bd750 service neutron] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:24.251280 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:24.256899 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5b244-4c9a-476a-8428-9ac27815c8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:24.257562 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5b244-4c9a-476a-8428-9ac27815c8a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:24.257855 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5b244-4c9a-476a-8428-9ac27815c8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:24.258132 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5b244-4c9a-476a-8428-9ac27815c8a0 admin admin] PATH_INFO: `/v3/users/018b805cff9946c3b5878df51e6332fe` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:24.259343 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf5b244-4c9a-476a-8428-9ac27815c8a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=018b805cff9946c3b5878df51e6332fe)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:24.261306 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf5b244-4c9a-476a-8428-9ac27815c8a0 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:24.530151 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-cdf5b244-4c9a-476a-8428-9ac27815c8a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '018b805cff9946c3b5878df51e6332fe'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:24.543306 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-cdf5b244-4c9a-476a-8428-9ac27815c8a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '018b805cff9946c3b5878df51e6332fe'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:24.545212 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-cdf5b244-4c9a-476a-8428-9ac27815c8a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '018b805cff9946c3b5878df51e6332fe'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:24.585251 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-cdf5b244-4c9a-476a-8428-9ac27815c8a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '018b805cff9946c3b5878df51e6332fe'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:24.667117 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-cdf5b244-4c9a-476a-8428-9ac27815c8a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '018b805cff9946c3b5878df51e6332fe'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:24.682167 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 843/1663] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:20:24 2022] DELETE /identity/v3/users/018b805cff9946c3b5878df51e6332fe => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:24.686533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04797b1-fd00-4cf1-9f4a-22c49817dccc None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:24.687068 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04797b1-fd00-4cf1-9f4a-22c49817dccc None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:24.687171 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04797b1-fd00-4cf1-9f4a-22c49817dccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:24.737931 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-e04797b1-fd00-4cf1-9f4a-22c49817dccc None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:24.742809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-e04797b1-fd00-4cf1-9f4a-22c49817dccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:24.908284 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 821/1664] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:24.939255 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5b244-4c9a-476a-8428-9ac27815c8a0 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:24.941041 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-cdf5b244-4c9a-476a-8428-9ac27815c8a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:25.085519 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5b244-4c9a-476a-8428-9ac27815c8a0 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:25.089958 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0055b0f-b77d-4eca-9006-aa2cd331535e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:25.090591 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0055b0f-b77d-4eca-9006-aa2cd331535e service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:25.090894 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0055b0f-b77d-4eca-9006-aa2cd331535e service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:25.091158 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0055b0f-b77d-4eca-9006-aa2cd331535e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:25.112108 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0055b0f-b77d-4eca-9006-aa2cd331535e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:25.123010 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04797b1-fd00-4cf1-9f4a-22c49817dccc None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:25.123917 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0055b0f-b77d-4eca-9006-aa2cd331535e service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:25.125039 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-e04797b1-fd00-4cf1-9f4a-22c49817dccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:25.149944 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04797b1-fd00-4cf1-9f4a-22c49817dccc None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:25.158085 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 844/1665] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:20:25.165727 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cdd25a-0ba3-4c94-98df-a5bcd3e8ccee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:25.166606 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdd25a-0ba3-4c94-98df-a5bcd3e8ccee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:25.166902 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdd25a-0ba3-4c94-98df-a5bcd3e8ccee admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:25.167317 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdd25a-0ba3-4c94-98df-a5bcd3e8ccee admin admin] PATH_INFO: `/v3/projects/74ec48e311334f6cbb2f9bcc1881b378` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:25.169486 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cdd25a-0ba3-4c94-98df-a5bcd3e8ccee admin admin] RBAC: Authorizing `identity:delete_project(project_id=74ec48e311334f6cbb2f9bcc1881b378)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:25.175273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cdd25a-0ba3-4c94-98df-a5bcd3e8ccee admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:25.278234 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0055b0f-b77d-4eca-9006-aa2cd331535e service neutron] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:25.279733 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-a0055b0f-b77d-4eca-9006-aa2cd331535e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:25.303589 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0055b0f-b77d-4eca-9006-aa2cd331535e service neutron] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:25.317152 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba7d01b-4acb-4ddb-b370-5fbc51d9bff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:25.317819 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7d01b-4acb-4ddb-b370-5fbc51d9bff4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:25.321183 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7d01b-4acb-4ddb-b370-5fbc51d9bff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:25.321183 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7d01b-4acb-4ddb-b370-5fbc51d9bff4 admin admin] PATH_INFO: `/v3/projects/296b5d138eff48958150d7dda904e484` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:25.321183 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba7d01b-4acb-4ddb-b370-5fbc51d9bff4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=296b5d138eff48958150d7dda904e484)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:25.321463 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba7d01b-4acb-4ddb-b370-5fbc51d9bff4 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:25.397043 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-a2cdd25a-0ba3-4c94-98df-a5bcd3e8ccee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74ec48e311334f6cbb2f9bcc1881b378', 'request_id': 'req-a2cdd25a-0ba3-4c94-98df-a5bcd3e8ccee'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:25.410663 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-a2cdd25a-0ba3-4c94-98df-a5bcd3e8ccee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74ec48e311334f6cbb2f9bcc1881b378', 'request_id': 'req-a2cdd25a-0ba3-4c94-98df-a5bcd3e8ccee'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:25.449829 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:25.507537 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-a2cdd25a-0ba3-4c94-98df-a5bcd3e8ccee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74ec48e311334f6cbb2f9bcc1881b378', 'request_id': 'req-a2cdd25a-0ba3-4c94-98df-a5bcd3e8ccee'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:25.525445 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-4ba7d01b-4acb-4ddb-b370-5fbc51d9bff4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '296b5d138eff48958150d7dda904e484', 'request_id': 'req-4ba7d01b-4acb-4ddb-b370-5fbc51d9bff4'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:25.531589 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-a2cdd25a-0ba3-4c94-98df-a5bcd3e8ccee admin admin] The token cache is being invalidate because project 74ec48e311334f6cbb2f9bcc1881b378 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:20:25.531931 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-a2cdd25a-0ba3-4c94-98df-a5bcd3e8ccee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:25.535268 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-a2cdd25a-0ba3-4c94-98df-a5bcd3e8ccee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:25.537710 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 822/1666] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:20:25 2022] DELETE /identity/v3/projects/74ec48e311334f6cbb2f9bcc1881b378 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:20:25.538955 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-4ba7d01b-4acb-4ddb-b370-5fbc51d9bff4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '296b5d138eff48958150d7dda904e484', 'request_id': 'req-4ba7d01b-4acb-4ddb-b370-5fbc51d9bff4'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:25.552523 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cdd25a-0ba3-4c94-98df-a5bcd3e8ccee admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:25.554518 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-a2cdd25a-0ba3-4c94-98df-a5bcd3e8ccee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:25.633359 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-4ba7d01b-4acb-4ddb-b370-5fbc51d9bff4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '296b5d138eff48958150d7dda904e484', 'request_id': 'req-4ba7d01b-4acb-4ddb-b370-5fbc51d9bff4'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:25.634944 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cdd25a-0ba3-4c94-98df-a5bcd3e8ccee admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:25.654293 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-4ba7d01b-4acb-4ddb-b370-5fbc51d9bff4 admin admin] The token cache is being invalidate because project 296b5d138eff48958150d7dda904e484 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:20:25.654596 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-4ba7d01b-4acb-4ddb-b370-5fbc51d9bff4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:25.656311 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-4ba7d01b-4acb-4ddb-b370-5fbc51d9bff4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:25.659453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 845/1667] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:20:25 2022] DELETE /identity/v3/projects/296b5d138eff48958150d7dda904e484 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:20:25.660304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba7d01b-4acb-4ddb-b370-5fbc51d9bff4 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:25.664606 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-4ba7d01b-4acb-4ddb-b370-5fbc51d9bff4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:25.779400 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c2d8f-9e4f-4072-b865-c271cdd2244f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:25.781281 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c2d8f-9e4f-4072-b865-c271cdd2244f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:25.781733 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c2d8f-9e4f-4072-b865-c271cdd2244f admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:25.782174 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c2d8f-9e4f-4072-b865-c271cdd2244f admin admin] PATH_INFO: `/v3/projects/eaa66bc30f22452e855c8d5a8a53e525` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:25.785438 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c2d8f-9e4f-4072-b865-c271cdd2244f admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaa66bc30f22452e855c8d5a8a53e525)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:25.791440 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c2d8f-9e4f-4072-b865-c271cdd2244f admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:25.792965 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba7d01b-4acb-4ddb-b370-5fbc51d9bff4 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:25.801006 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22816aba-e0d2-4624-80c3-d1e33b5d9177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:25.801681 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22816aba-e0d2-4624-80c3-d1e33b5d9177 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:25.803467 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22816aba-e0d2-4624-80c3-d1e33b5d9177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:25.803796 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22816aba-e0d2-4624-80c3-d1e33b5d9177 admin admin] PATH_INFO: `/v3/users/8134ec0bede6493e8c8e8e9ce2a5b800` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:25.809824 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22816aba-e0d2-4624-80c3-d1e33b5d9177 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8134ec0bede6493e8c8e8e9ce2a5b800)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:25.816915 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22816aba-e0d2-4624-80c3-d1e33b5d9177 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:26.056034 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-22816aba-e0d2-4624-80c3-d1e33b5d9177 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8134ec0bede6493e8c8e8e9ce2a5b800'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:26.057997 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-6c4c2d8f-9e4f-4072-b865-c271cdd2244f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaa66bc30f22452e855c8d5a8a53e525', 'request_id': 'req-6c4c2d8f-9e4f-4072-b865-c271cdd2244f'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:26.072035 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-6c4c2d8f-9e4f-4072-b865-c271cdd2244f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaa66bc30f22452e855c8d5a8a53e525', 'request_id': 'req-6c4c2d8f-9e4f-4072-b865-c271cdd2244f'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:26.075553 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-22816aba-e0d2-4624-80c3-d1e33b5d9177 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8134ec0bede6493e8c8e8e9ce2a5b800'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:26.076356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-22816aba-e0d2-4624-80c3-d1e33b5d9177 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8134ec0bede6493e8c8e8e9ce2a5b800'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:26.101529 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-22816aba-e0d2-4624-80c3-d1e33b5d9177 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8134ec0bede6493e8c8e8e9ce2a5b800'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:26.143378 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-6c4c2d8f-9e4f-4072-b865-c271cdd2244f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaa66bc30f22452e855c8d5a8a53e525', 'request_id': 'req-6c4c2d8f-9e4f-4072-b865-c271cdd2244f'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:26.167290 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-6c4c2d8f-9e4f-4072-b865-c271cdd2244f admin admin] The token cache is being invalidate because project eaa66bc30f22452e855c8d5a8a53e525 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:20:26.168054 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-6c4c2d8f-9e4f-4072-b865-c271cdd2244f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:26.169647 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-6c4c2d8f-9e4f-4072-b865-c271cdd2244f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:26.172131 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 823/1668] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:20:25 2022] DELETE /identity/v3/projects/eaa66bc30f22452e855c8d5a8a53e525 => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:26.173864 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c2d8f-9e4f-4072-b865-c271cdd2244f admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:26.177140 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-6c4c2d8f-9e4f-4072-b865-c271cdd2244f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:26.224534 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:26.226468 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-22816aba-e0d2-4624-80c3-d1e33b5d9177 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8134ec0bede6493e8c8e8e9ce2a5b800'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:26.245573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 846/1669] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:20:25 2022] DELETE /identity/v3/users/8134ec0bede6493e8c8e8e9ce2a5b800 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:26.251709 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2a434d-0371-4a01-83c3-2a0356d75b51 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:26.253941 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2a434d-0371-4a01-83c3-2a0356d75b51 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:26.255190 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2a434d-0371-4a01-83c3-2a0356d75b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:26.332221 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-3b2a434d-0371-4a01-83c3-2a0356d75b51 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:26.334199 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-3b2a434d-0371-4a01-83c3-2a0356d75b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:26.417867 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c2d8f-9e4f-4072-b865-c271cdd2244f admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:26.551376 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3a033-3e5b-4992-a923-c77a4d9490b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:26.552257 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3a033-3e5b-4992-a923-c77a4d9490b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:26.552634 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3a033-3e5b-4992-a923-c77a4d9490b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:26.553011 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3a033-3e5b-4992-a923-c77a4d9490b2 admin admin] PATH_INFO: `/v3/users/3e5b3f0e256a428cba079f95d2c9a0ee` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:26.554402 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3a033-3e5b-4992-a923-c77a4d9490b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e5b3f0e256a428cba079f95d2c9a0ee)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:26.557194 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3a033-3e5b-4992-a923-c77a4d9490b2 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:26.594827 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 847/1670] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:26.594827 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2a434d-0371-4a01-83c3-2a0356d75b51 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:26.594827 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-3b2a434d-0371-4a01-83c3-2a0356d75b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:26.664202 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2a434d-0371-4a01-83c3-2a0356d75b51 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:26.679863 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae1c63-cbbe-49f8-b0ea-6f1ce0a57a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:26.680670 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae1c63-cbbe-49f8-b0ea-6f1ce0a57a5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:26.681108 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae1c63-cbbe-49f8-b0ea-6f1ce0a57a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:26.681489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae1c63-cbbe-49f8-b0ea-6f1ce0a57a5c admin admin] PATH_INFO: `/v3/projects/69a97ad9036941329090a7723cac0cb4` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:26.682853 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dae1c63-cbbe-49f8-b0ea-6f1ce0a57a5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=69a97ad9036941329090a7723cac0cb4)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:26.684926 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dae1c63-cbbe-49f8-b0ea-6f1ce0a57a5c admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:26.841510 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-0dae1c63-cbbe-49f8-b0ea-6f1ce0a57a5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69a97ad9036941329090a7723cac0cb4', 'request_id': 'req-0dae1c63-cbbe-49f8-b0ea-6f1ce0a57a5c'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:26.845011 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-93e3a033-3e5b-4992-a923-c77a4d9490b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e5b3f0e256a428cba079f95d2c9a0ee'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:26.849072 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-93e3a033-3e5b-4992-a923-c77a4d9490b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e5b3f0e256a428cba079f95d2c9a0ee'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:26.852520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-93e3a033-3e5b-4992-a923-c77a4d9490b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e5b3f0e256a428cba079f95d2c9a0ee'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:26.884939 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-0dae1c63-cbbe-49f8-b0ea-6f1ce0a57a5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69a97ad9036941329090a7723cac0cb4', 'request_id': 'req-0dae1c63-cbbe-49f8-b0ea-6f1ce0a57a5c'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:26.887011 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-93e3a033-3e5b-4992-a923-c77a4d9490b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e5b3f0e256a428cba079f95d2c9a0ee'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:26.940413 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-93e3a033-3e5b-4992-a923-c77a4d9490b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e5b3f0e256a428cba079f95d2c9a0ee'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:27.004036 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-0dae1c63-cbbe-49f8-b0ea-6f1ce0a57a5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69a97ad9036941329090a7723cac0cb4', 'request_id': 'req-0dae1c63-cbbe-49f8-b0ea-6f1ce0a57a5c'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:27.038905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-0dae1c63-cbbe-49f8-b0ea-6f1ce0a57a5c admin admin] The token cache is being invalidate because project 69a97ad9036941329090a7723cac0cb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:20:27.041101 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-0dae1c63-cbbe-49f8-b0ea-6f1ce0a57a5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:27.043084 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-0dae1c63-cbbe-49f8-b0ea-6f1ce0a57a5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:27.051000 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 848/1671] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:20:26 2022] DELETE /identity/v3/projects/69a97ad9036941329090a7723cac0cb4 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:27.052575 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4da34-2713-4df8-80a3-ea391cfa307a None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:27.055184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4da34-2713-4df8-80a3-ea391cfa307a None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:27.055536 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4da34-2713-4df8-80a3-ea391cfa307a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:27.099504 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 824/1672] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:20:26 2022] DELETE /identity/v3/users/3e5b3f0e256a428cba079f95d2c9a0ee => generated 0 bytes in 925 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:27.102365 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53913ec-4dec-46f0-98cc-8ee93c86a1e6 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:27.102365 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53913ec-4dec-46f0-98cc-8ee93c86a1e6 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:27.102365 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53913ec-4dec-46f0-98cc-8ee93c86a1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:27.125545 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-50f4da34-2713-4df8-80a3-ea391cfa307a None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:27.125545 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-50f4da34-2713-4df8-80a3-ea391cfa307a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:27.178230 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-e53913ec-4dec-46f0-98cc-8ee93c86a1e6 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:27.179297 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-e53913ec-4dec-46f0-98cc-8ee93c86a1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:27.368650 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 849/1673] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:27.373424 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f4da34-2713-4df8-80a3-ea391cfa307a None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:27.395540 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f4da34-2713-4df8-80a3-ea391cfa307a None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:27.411076 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759225b3-1283-4e2c-bd40-c67d2f9ed958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:27.411757 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759225b3-1283-4e2c-bd40-c67d2f9ed958 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:27.412044 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759225b3-1283-4e2c-bd40-c67d2f9ed958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:27.412310 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759225b3-1283-4e2c-bd40-c67d2f9ed958 admin admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:27.413020 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759225b3-1283-4e2c-bd40-c67d2f9ed958 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:20:27.413392 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759225b3-1283-4e2c-bd40-c67d2f9ed958 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:27.415055 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759225b3-1283-4e2c-bd40-c67d2f9ed958 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:27.432720 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:27.449272 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 850/1674] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:20:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:20:27.450430 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759225b3-1283-4e2c-bd40-c67d2f9ed958 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:27.478957 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 825/1675] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:27.479767 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759225b3-1283-4e2c-bd40-c67d2f9ed958 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:27.486226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df02743-4e14-4186-88a8-03cefd50c95e None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:27.492999 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df02743-4e14-4186-88a8-03cefd50c95e None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:27.492999 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df02743-4e14-4186-88a8-03cefd50c95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:27.495523 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177fead7-283a-49e4-b643-dfd2d8dfcf02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:27.496206 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fead7-283a-49e4-b643-dfd2d8dfcf02 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:27.496489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fead7-283a-49e4-b643-dfd2d8dfcf02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:27.496764 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fead7-283a-49e4-b643-dfd2d8dfcf02 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:27.499712 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177fead7-283a-49e4-b643-dfd2d8dfcf02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:27.505706 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177fead7-283a-49e4-b643-dfd2d8dfcf02 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:27.574977 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-7df02743-4e14-4186-88a8-03cefd50c95e None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:27.580707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-7df02743-4e14-4186-88a8-03cefd50c95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:27.607435 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 826/1676] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:27.610400 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df02743-4e14-4186-88a8-03cefd50c95e None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:27.631690 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df02743-4e14-4186-88a8-03cefd50c95e None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:27.646387 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c6d0b3-ecf2-4c64-8e06-2500c5363b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:27.649151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6d0b3-ecf2-4c64-8e06-2500c5363b4d admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:27.649706 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6d0b3-ecf2-4c64-8e06-2500c5363b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:27.650983 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6d0b3-ecf2-4c64-8e06-2500c5363b4d admin admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:27.652909 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c6d0b3-ecf2-4c64-8e06-2500c5363b4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:20:27.653780 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c6d0b3-ecf2-4c64-8e06-2500c5363b4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:27.656035 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c6d0b3-ecf2-4c64-8e06-2500c5363b4d admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:27.677452 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 851/1677] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:20:27 2022] POST /identity/v3/projects => generated 371 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:27.681397 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177fead7-283a-49e4-b643-dfd2d8dfcf02 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:27.708641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 827/1678] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:20:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:20:27.717258 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c6d0b3-ecf2-4c64-8e06-2500c5363b4d admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:27.737281 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c6d0b3-ecf2-4c64-8e06-2500c5363b4d admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:27.860869 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177fead7-283a-49e4-b643-dfd2d8dfcf02 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:27.876568 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b696dcb-7747-4906-89bb-001e1ae3f5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:27.877237 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b696dcb-7747-4906-89bb-001e1ae3f5a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:27.877802 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b696dcb-7747-4906-89bb-001e1ae3f5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:27.877919 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b696dcb-7747-4906-89bb-001e1ae3f5a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:27.878535 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b696dcb-7747-4906-89bb-001e1ae3f5a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:27.881368 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b696dcb-7747-4906-89bb-001e1ae3f5a1 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:27.894700 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6294c2c-85cb-4833-a2a7-96932e0011c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:27.896399 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6294c2c-85cb-4833-a2a7-96932e0011c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:27.899529 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6294c2c-85cb-4833-a2a7-96932e0011c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:27.899529 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6294c2c-85cb-4833-a2a7-96932e0011c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:27.900922 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6294c2c-85cb-4833-a2a7-96932e0011c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:27.904346 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6294c2c-85cb-4833-a2a7-96932e0011c9 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:28.032320 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 852/1679] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:20:27 2022] POST /identity/v3/users => generated 286 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:28.036149 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b696dcb-7747-4906-89bb-001e1ae3f5a1 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:28.063206 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 828/1680] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:20:27 2022] POST /identity/v3/projects => generated 365 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:28.063285 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b696dcb-7747-4906-89bb-001e1ae3f5a1 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:28.064606 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6294c2c-85cb-4833-a2a7-96932e0011c9 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:28.167241 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:28.208121 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d69666-a4ec-408a-b524-d66e7da07f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:28.208762 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d69666-a4ec-408a-b524-d66e7da07f7f admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:28.209046 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d69666-a4ec-408a-b524-d66e7da07f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:28.209323 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d69666-a4ec-408a-b524-d66e7da07f7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:28.209988 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d69666-a4ec-408a-b524-d66e7da07f7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:28.217592 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d69666-a4ec-408a-b524-d66e7da07f7f admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:28.261873 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 853/1681] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:28 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 12:20:28.261873 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d69666-a4ec-408a-b524-d66e7da07f7f admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:28.278664 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6294c2c-85cb-4833-a2a7-96932e0011c9 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:28.300572 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d69666-a4ec-408a-b524-d66e7da07f7f admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:28.312588 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ed7dc-9221-43b0-9215-539a41151256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:28.313347 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ed7dc-9221-43b0-9215-539a41151256 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:28.313574 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ed7dc-9221-43b0-9215-539a41151256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:28.313854 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ed7dc-9221-43b0-9215-539a41151256 admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:28.314220 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb829d14-1e7c-4e12-aad0-59e2d3a6f2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:28.314505 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4ed7dc-9221-43b0-9215-539a41151256 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:28.315566 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb829d14-1e7c-4e12-aad0-59e2d3a6f2b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:28.316046 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb829d14-1e7c-4e12-aad0-59e2d3a6f2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:28.316918 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb829d14-1e7c-4e12-aad0-59e2d3a6f2b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:28.317953 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb829d14-1e7c-4e12-aad0-59e2d3a6f2b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:28.319805 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4ed7dc-9221-43b0-9215-539a41151256 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:28.320596 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb829d14-1e7c-4e12-aad0-59e2d3a6f2b3 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:28.341808 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 854/1682] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:28 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:28.349610 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb829d14-1e7c-4e12-aad0-59e2d3a6f2b3 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:28.380687 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb829d14-1e7c-4e12-aad0-59e2d3a6f2b3 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:28.398918 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce5b0a-10c2-4baf-889b-4df3df2199d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:28.403274 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce5b0a-10c2-4baf-889b-4df3df2199d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:28.403274 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce5b0a-10c2-4baf-889b-4df3df2199d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:28.403274 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce5b0a-10c2-4baf-889b-4df3df2199d6 admin admin] PATH_INFO: `/v3/projects/f541fae8121b499d93c22d2870afb56f/users/be651e0d775448d2914ca364904dfd5b/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:28.428488 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 829/1683] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:20:28 2022] POST /identity/v3/users => generated 283 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:28.432597 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ed7dc-9221-43b0-9215-539a41151256 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:28.434257 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce5b0a-10c2-4baf-889b-4df3df2199d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f541fae8121b499d93c22d2870afb56f, user_id=be651e0d775448d2914ca364904dfd5b, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:28.437749 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce5b0a-10c2-4baf-889b-4df3df2199d6 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:28.459877 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ed7dc-9221-43b0-9215-539a41151256 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:28.471078 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b6321-1afe-44c6-83e5-6e2e78097270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:28.472468 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b6321-1afe-44c6-83e5-6e2e78097270 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:28.477727 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b6321-1afe-44c6-83e5-6e2e78097270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:28.477727 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b6321-1afe-44c6-83e5-6e2e78097270 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:28.477727 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7b6321-1afe-44c6-83e5-6e2e78097270 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:28.479523 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7b6321-1afe-44c6-83e5-6e2e78097270 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:28.506929 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 830/1684] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:28 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:28.508530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b6321-1afe-44c6-83e5-6e2e78097270 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:28.522040 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b6321-1afe-44c6-83e5-6e2e78097270 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:28.533130 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d16b28a-587a-4531-96bd-8432f0e19f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:28.533746 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16b28a-587a-4531-96bd-8432f0e19f01 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:28.534285 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 855/1685] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:20:28 2022] PUT /identity/v3/projects/f541fae8121b499d93c22d2870afb56f/users/be651e0d775448d2914ca364904dfd5b/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:28.545810 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c87aee-4218-4c72-8a9d-8df2cfa58ed1 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:28.546129 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c87aee-4218-4c72-8a9d-8df2cfa58ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:28.546408 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c87aee-4218-4c72-8a9d-8df2cfa58ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:28.547077 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16b28a-587a-4531-96bd-8432f0e19f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:28.547369 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16b28a-587a-4531-96bd-8432f0e19f01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:28.548157 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d16b28a-587a-4531-96bd-8432f0e19f01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:28.549759 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d16b28a-587a-4531-96bd-8432f0e19f01 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:28.575258 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 831/1686] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:28 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:28.580291 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d16b28a-587a-4531-96bd-8432f0e19f01 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:28.605084 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-17c87aee-4218-4c72-8a9d-8df2cfa58ed1 None None] MFA Rules not processed for user `be651e0d775448d2914ca364904dfd5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:28.605365 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d16b28a-587a-4531-96bd-8432f0e19f01 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:28.606312 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-17c87aee-4218-4c72-8a9d-8df2cfa58ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:28.738210 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:28.763900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a2804-4eab-41d9-b2be-deae0f00ad01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:28.765509 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a2804-4eab-41d9-b2be-deae0f00ad01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:28.765950 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a2804-4eab-41d9-b2be-deae0f00ad01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:28.766465 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a2804-4eab-41d9-b2be-deae0f00ad01 admin admin] PATH_INFO: `/v3/projects/d539003295e34a98bef641ed4186686c/users/432509e0a09b40a0b4366334e7c229d5/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:28.786583 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8a2804-4eab-41d9-b2be-deae0f00ad01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d539003295e34a98bef641ed4186686c, user_id=432509e0a09b40a0b4366334e7c229d5, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:28.790337 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8a2804-4eab-41d9-b2be-deae0f00ad01 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:28.800696 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 856/1687] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:20:28.808472 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c87aee-4218-4c72-8a9d-8df2cfa58ed1 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:28.808886 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-17c87aee-4218-4c72-8a9d-8df2cfa58ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:28.831229 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c87aee-4218-4c72-8a9d-8df2cfa58ed1 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:28.838037 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1d35a6-8f45-445f-a562-24b50d070800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:28.838775 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d35a6-8f45-445f-a562-24b50d070800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:28.839246 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d35a6-8f45-445f-a562-24b50d070800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:28.841919 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d35a6-8f45-445f-a562-24b50d070800 admin admin] PATH_INFO: `/v3/projects/0a3aed9d3ac44f0287f88dc207cd9c14` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:28.843694 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1d35a6-8f45-445f-a562-24b50d070800 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a3aed9d3ac44f0287f88dc207cd9c14)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:28.846160 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1d35a6-8f45-445f-a562-24b50d070800 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:28.892090 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 832/1688] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:20:28 2022] PUT /identity/v3/projects/d539003295e34a98bef641ed4186686c/users/432509e0a09b40a0b4366334e7c229d5/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:20:28.893071 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a2804-4eab-41d9-b2be-deae0f00ad01 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:29.054333 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-9f1d35a6-8f45-445f-a562-24b50d070800 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a3aed9d3ac44f0287f88dc207cd9c14', 'request_id': 'req-9f1d35a6-8f45-445f-a562-24b50d070800'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:29.080160 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-9f1d35a6-8f45-445f-a562-24b50d070800 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a3aed9d3ac44f0287f88dc207cd9c14', 'request_id': 'req-9f1d35a6-8f45-445f-a562-24b50d070800'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:29.103759 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a2804-4eab-41d9-b2be-deae0f00ad01 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:29.160198 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-9f1d35a6-8f45-445f-a562-24b50d070800 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a3aed9d3ac44f0287f88dc207cd9c14', 'request_id': 'req-9f1d35a6-8f45-445f-a562-24b50d070800'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:29.193718 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-9f1d35a6-8f45-445f-a562-24b50d070800 admin admin] The token cache is being invalidate because project 0a3aed9d3ac44f0287f88dc207cd9c14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:20:29.194017 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-9f1d35a6-8f45-445f-a562-24b50d070800 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:29.197472 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-9f1d35a6-8f45-445f-a562-24b50d070800 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:29.210892 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 857/1689] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:20:28 2022] DELETE /identity/v3/projects/0a3aed9d3ac44f0287f88dc207cd9c14 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:29.216650 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e0852-c9b0-44a8-920e-55cdf0007ee8 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:29.216650 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e0852-c9b0-44a8-920e-55cdf0007ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:29.216650 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e0852-c9b0-44a8-920e-55cdf0007ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:29.248335 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21f0c2-e958-44d0-ac5d-53a625582d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:29.249181 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21f0c2-e958-44d0-ac5d-53a625582d64 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:29.249712 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21f0c2-e958-44d0-ac5d-53a625582d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:29.250437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21f0c2-e958-44d0-ac5d-53a625582d64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:29.250863 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc21f0c2-e958-44d0-ac5d-53a625582d64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:29.253163 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc21f0c2-e958-44d0-ac5d-53a625582d64 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:29.360132 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-fd7e0852-c9b0-44a8-920e-55cdf0007ee8 None None] MFA Rules not processed for user `432509e0a09b40a0b4366334e7c229d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:29.364165 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 833/1690] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:20:28 2022] POST /identity/v3/projects => generated 371 bytes in 471 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:29.365369 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21f0c2-e958-44d0-ac5d-53a625582d64 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:29.375320 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-fd7e0852-c9b0-44a8-920e-55cdf0007ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:29.376313 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-fc21f0c2-e958-44d0-ac5d-53a625582d64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:29.528204 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21f0c2-e958-44d0-ac5d-53a625582d64 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:29.544978 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70861123-b6e4-429a-b15f-d7ed0452f73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:29.545668 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70861123-b6e4-429a-b15f-d7ed0452f73e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:29.545965 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70861123-b6e4-429a-b15f-d7ed0452f73e admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:29.546243 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70861123-b6e4-429a-b15f-d7ed0452f73e admin admin] PATH_INFO: `/v3/projects/bdf7b54a8ed14ca7b73c7972b039309f` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:29.550276 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70861123-b6e4-429a-b15f-d7ed0452f73e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdf7b54a8ed14ca7b73c7972b039309f)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:29.558882 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70861123-b6e4-429a-b15f-d7ed0452f73e admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:29.581258 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:29.589084 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 858/1691] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:29.591021 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77bd867-e50e-4f5f-98b5-3cdbabce9b2b None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:29.591327 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77bd867-e50e-4f5f-98b5-3cdbabce9b2b None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:29.591634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77bd867-e50e-4f5f-98b5-3cdbabce9b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:29.639032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-f77bd867-e50e-4f5f-98b5-3cdbabce9b2b None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:29.642139 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-f77bd867-e50e-4f5f-98b5-3cdbabce9b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:29.687955 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 859/1692] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:29.688982 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77bd867-e50e-4f5f-98b5-3cdbabce9b2b None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:29.694224 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-f77bd867-e50e-4f5f-98b5-3cdbabce9b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:29.726083 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77bd867-e50e-4f5f-98b5-3cdbabce9b2b None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:29.789707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-70861123-b6e4-429a-b15f-d7ed0452f73e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdf7b54a8ed14ca7b73c7972b039309f', 'request_id': 'req-70861123-b6e4-429a-b15f-d7ed0452f73e'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:29.805508 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-70861123-b6e4-429a-b15f-d7ed0452f73e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdf7b54a8ed14ca7b73c7972b039309f', 'request_id': 'req-70861123-b6e4-429a-b15f-d7ed0452f73e'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:29.881343 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d6237-be06-48f8-b8fb-a7e9ba4e7460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:29.881343 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d6237-be06-48f8-b8fb-a7e9ba4e7460 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:29.882196 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d6237-be06-48f8-b8fb-a7e9ba4e7460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:29.882512 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d6237-be06-48f8-b8fb-a7e9ba4e7460 admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:29.883204 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d6237-be06-48f8-b8fb-a7e9ba4e7460 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:29.885244 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d6237-be06-48f8-b8fb-a7e9ba4e7460 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:29.962998 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-70861123-b6e4-429a-b15f-d7ed0452f73e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdf7b54a8ed14ca7b73c7972b039309f', 'request_id': 'req-70861123-b6e4-429a-b15f-d7ed0452f73e'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:29.988177 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-70861123-b6e4-429a-b15f-d7ed0452f73e admin admin] The token cache is being invalidate because project bdf7b54a8ed14ca7b73c7972b039309f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:20:29.993489 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-70861123-b6e4-429a-b15f-d7ed0452f73e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:29.993489 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-70861123-b6e4-429a-b15f-d7ed0452f73e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:30.000869 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 834/1693] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:20:29 2022] DELETE /identity/v3/projects/bdf7b54a8ed14ca7b73c7972b039309f => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:30.001040 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 860/1694] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:20:29 2022] POST /identity/v3/users => generated 292 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:30.001115 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70861123-b6e4-429a-b15f-d7ed0452f73e admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:30.001181 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2571ca-fa06-42f8-bd9c-2aa16d4ecd37 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:30.001181 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2571ca-fa06-42f8-bd9c-2aa16d4ecd37 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:30.001181 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2571ca-fa06-42f8-bd9c-2aa16d4ecd37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:30.003448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-70861123-b6e4-429a-b15f-d7ed0452f73e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:30.020548 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70861123-b6e4-429a-b15f-d7ed0452f73e admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:30.027620 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642cc3fe-58e5-4d3e-807d-dddb8e59b288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:30.027620 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cc3fe-58e5-4d3e-807d-dddb8e59b288 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:30.027620 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cc3fe-58e5-4d3e-807d-dddb8e59b288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:30.027620 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cc3fe-58e5-4d3e-807d-dddb8e59b288 admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:30.028164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642cc3fe-58e5-4d3e-807d-dddb8e59b288 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:30.030073 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642cc3fe-58e5-4d3e-807d-dddb8e59b288 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:30.044525 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-bd2571ca-fa06-42f8-bd9c-2aa16d4ecd37 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:30.050565 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-bd2571ca-fa06-42f8-bd9c-2aa16d4ecd37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:30.068437 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 861/1695] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:30.069578 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2571ca-fa06-42f8-bd9c-2aa16d4ecd37 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:30.071204 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-bd2571ca-fa06-42f8-bd9c-2aa16d4ecd37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:30.110524 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2571ca-fa06-42f8-bd9c-2aa16d4ecd37 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:30.125117 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4123e0b1-dc1e-4db4-ae6b-b82c5b3f08da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:30.125117 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123e0b1-dc1e-4db4-ae6b-b82c5b3f08da admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:30.125117 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123e0b1-dc1e-4db4-ae6b-b82c5b3f08da admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:30.125117 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123e0b1-dc1e-4db4-ae6b-b82c5b3f08da admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:30.126098 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4123e0b1-dc1e-4db4-ae6b-b82c5b3f08da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:30.126098 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4123e0b1-dc1e-4db4-ae6b-b82c5b3f08da admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:30.146618 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 835/1696] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:20:29 2022] POST /identity/v3/projects => generated 363 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:30.149751 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df3e4e-d29c-4f6e-96d6-c86b0b42c9ba None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:30.150293 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df3e4e-d29c-4f6e-96d6-c86b0b42c9ba None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:30.151104 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df3e4e-d29c-4f6e-96d6-c86b0b42c9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:30.152192 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 862/1697] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:30.162073 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4123e0b1-dc1e-4db4-ae6b-b82c5b3f08da admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:30.179729 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-57df3e4e-d29c-4f6e-96d6-c86b0b42c9ba None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:30.182159 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-57df3e4e-d29c-4f6e-96d6-c86b0b42c9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:30.183138 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4123e0b1-dc1e-4db4-ae6b-b82c5b3f08da admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:30.206537 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:30.300083 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 836/1698] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:30.300083 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df3e4e-d29c-4f6e-96d6-c86b0b42c9ba None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:30.308369 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df3e4e-d29c-4f6e-96d6-c86b0b42c9ba None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:30.314795 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adc8b56-4266-41d0-86b8-a3e9e204e806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:30.315812 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc8b56-4266-41d0-86b8-a3e9e204e806 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:30.316255 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc8b56-4266-41d0-86b8-a3e9e204e806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:30.316663 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc8b56-4266-41d0-86b8-a3e9e204e806 admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:30.317437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adc8b56-4266-41d0-86b8-a3e9e204e806 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:30.319619 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adc8b56-4266-41d0-86b8-a3e9e204e806 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:30.329805 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e49ca22-b1a9-446f-86b1-00988e02e327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:30.330469 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49ca22-b1a9-446f-86b1-00988e02e327 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:30.330758 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49ca22-b1a9-446f-86b1-00988e02e327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:30.331037 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49ca22-b1a9-446f-86b1-00988e02e327 admin admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:30.331749 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e49ca22-b1a9-446f-86b1-00988e02e327 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:20:30.332045 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e49ca22-b1a9-446f-86b1-00988e02e327 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:30.343817 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e49ca22-b1a9-446f-86b1-00988e02e327 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:30.363297 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 863/1699] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:20:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:20:30.364693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e49ca22-b1a9-446f-86b1-00988e02e327 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:30.385170 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e49ca22-b1a9-446f-86b1-00988e02e327 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:30.435177 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f468ccb-431f-402b-b7a7-0b306da78640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:30.436285 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f468ccb-431f-402b-b7a7-0b306da78640 admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:30.442086 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 837/1700] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:20:30 2022] POST /identity/v3/users => generated 288 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:30.442232 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f468ccb-431f-402b-b7a7-0b306da78640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:30.442232 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f468ccb-431f-402b-b7a7-0b306da78640 admin admin] PATH_INFO: `/v3/projects/21fbb46ad3a949da935fd35b22bcc60f/users/152c1070e5964c469d2790f2cfe85236/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:30.442358 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b26c75-7922-485e-b943-2e09c87cc282 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:30.442358 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b26c75-7922-485e-b943-2e09c87cc282 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:30.442358 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b26c75-7922-485e-b943-2e09c87cc282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:30.497138 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f468ccb-431f-402b-b7a7-0b306da78640 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21fbb46ad3a949da935fd35b22bcc60f, user_id=152c1070e5964c469d2790f2cfe85236, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:30.508892 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f468ccb-431f-402b-b7a7-0b306da78640 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:30.522263 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-28b26c75-7922-485e-b943-2e09c87cc282 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:30.528093 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-28b26c75-7922-485e-b943-2e09c87cc282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:30.557164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 838/1701] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:30.558035 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b26c75-7922-485e-b943-2e09c87cc282 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:30.584659 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b26c75-7922-485e-b943-2e09c87cc282 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:30.601987 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e31e774-5b62-411c-bdfb-dd208979843c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:30.601987 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31e774-5b62-411c-bdfb-dd208979843c admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:30.601987 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31e774-5b62-411c-bdfb-dd208979843c admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:30.601987 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31e774-5b62-411c-bdfb-dd208979843c admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:30.602511 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e31e774-5b62-411c-bdfb-dd208979843c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:30.604491 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e31e774-5b62-411c-bdfb-dd208979843c admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:30.626497 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 864/1702] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:20:30 2022] PUT /identity/v3/projects/21fbb46ad3a949da935fd35b22bcc60f/users/152c1070e5964c469d2790f2cfe85236/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:30.628075 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f468ccb-431f-402b-b7a7-0b306da78640 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:30.784493 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 839/1703] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:20:30 2022] POST /identity/v3/projects => generated 371 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:30.784493 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e31e774-5b62-411c-bdfb-dd208979843c admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:30.832930 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f468ccb-431f-402b-b7a7-0b306da78640 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:30.941157 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e31e774-5b62-411c-bdfb-dd208979843c admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:30.959379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b0f0ff-887d-497f-87b4-dcb355ae8bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:30.967110 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0f0ff-887d-497f-87b4-dcb355ae8bfb admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:30.967555 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0f0ff-887d-497f-87b4-dcb355ae8bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:30.968401 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0f0ff-887d-497f-87b4-dcb355ae8bfb admin admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:30.974709 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b0f0ff-887d-497f-87b4-dcb355ae8bfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:20:30.974709 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b0f0ff-887d-497f-87b4-dcb355ae8bfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:30.976157 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b0f0ff-887d-497f-87b4-dcb355ae8bfb admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:30.986225 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efec04c-ea72-4d7b-af63-96052c9d6984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:30.986910 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efec04c-ea72-4d7b-af63-96052c9d6984 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:30.987228 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efec04c-ea72-4d7b-af63-96052c9d6984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:30.987495 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efec04c-ea72-4d7b-af63-96052c9d6984 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:30.988295 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efec04c-ea72-4d7b-af63-96052c9d6984 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:30.997691 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efec04c-ea72-4d7b-af63-96052c9d6984 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:31.033566 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 840/1704] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:20:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:20:31.034936 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 865/1705] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:31.036060 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efec04c-ea72-4d7b-af63-96052c9d6984 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:31.048623 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb8917-74cc-4100-b073-cadb983dd04c None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:31.051843 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb8917-74cc-4100-b073-cadb983dd04c None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:31.052318 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb8917-74cc-4100-b073-cadb983dd04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:31.066631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efec04c-ea72-4d7b-af63-96052c9d6984 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:31.076967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2d488-ce48-4b24-a600-e5981bb97cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:31.078273 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2d488-ce48-4b24-a600-e5981bb97cec admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:31.078646 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2d488-ce48-4b24-a600-e5981bb97cec admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:31.078981 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2d488-ce48-4b24-a600-e5981bb97cec admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:31.079751 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db2d488-ce48-4b24-a600-e5981bb97cec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:31.081521 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db2d488-ce48-4b24-a600-e5981bb97cec admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:31.094975 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 866/1706] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:31.096182 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2d488-ce48-4b24-a600-e5981bb97cec admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:31.118704 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-6cfb8917-74cc-4100-b073-cadb983dd04c None None] MFA Rules not processed for user `b5113b1a77fd4b5da175b29b40d2b636`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:31.125110 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2d488-ce48-4b24-a600-e5981bb97cec admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:31.126431 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-6cfb8917-74cc-4100-b073-cadb983dd04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:31.137680 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7e026-6b6c-4e48-b9a4-409984800f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:31.138324 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7e026-6b6c-4e48-b9a4-409984800f26 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:31.138615 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7e026-6b6c-4e48-b9a4-409984800f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:31.140626 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7e026-6b6c-4e48-b9a4-409984800f26 admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:31.141319 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e7e026-6b6c-4e48-b9a4-409984800f26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:31.143233 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e7e026-6b6c-4e48-b9a4-409984800f26 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:31.160362 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:31.256368 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 841/1707] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:31.256368 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d89677-41c5-49bd-b61f-090f55d9e88c None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:31.256368 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d89677-41c5-49bd-b61f-090f55d9e88c None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:31.256368 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d89677-41c5-49bd-b61f-090f55d9e88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:31.283656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 867/1708] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:20:31 2022] POST /identity/v3/users => generated 286 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:31.285348 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7e026-6b6c-4e48-b9a4-409984800f26 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:31.290208 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-a1d89677-41c5-49bd-b61f-090f55d9e88c None None] MFA Rules not processed for user `2e8380a466484c83bca842518d171048`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:31.291576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-a1d89677-41c5-49bd-b61f-090f55d9e88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:31.311363 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7e026-6b6c-4e48-b9a4-409984800f26 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:31.320596 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854b280-f611-4269-a72e-094e6900c041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:31.322516 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854b280-f611-4269-a72e-094e6900c041 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:31.322689 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854b280-f611-4269-a72e-094e6900c041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:31.323033 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854b280-f611-4269-a72e-094e6900c041 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:31.323816 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1854b280-f611-4269-a72e-094e6900c041 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:31.326643 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1854b280-f611-4269-a72e-094e6900c041 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:31.388553 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 842/1709] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:31 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:31.390268 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d89677-41c5-49bd-b61f-090f55d9e88c None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:31.407167 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d89677-41c5-49bd-b61f-090f55d9e88c None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:31.412633 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580d7e7-33a3-40ff-9398-2e6a64b96c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:31.413699 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580d7e7-33a3-40ff-9398-2e6a64b96c00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:31.414166 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580d7e7-33a3-40ff-9398-2e6a64b96c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:31.414805 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580d7e7-33a3-40ff-9398-2e6a64b96c00 admin admin] PATH_INFO: `/v3/projects/1e4ad0eb37e4424d8fd4492518a88369/users/9aea0547b9334314bd61be0f6cf90299/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:31.430579 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 868/1710] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:20:31 2022] POST /identity/v3/projects => generated 387 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:31.431360 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854b280-f611-4269-a72e-094e6900c041 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:31.443034 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c580d7e7-33a3-40ff-9398-2e6a64b96c00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e4ad0eb37e4424d8fd4492518a88369, user_id=9aea0547b9334314bd61be0f6cf90299, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:31.449722 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c580d7e7-33a3-40ff-9398-2e6a64b96c00 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:31.526244 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854b280-f611-4269-a72e-094e6900c041 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:31.531580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 843/1711] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:20:31 2022] PUT /identity/v3/projects/1e4ad0eb37e4424d8fd4492518a88369/users/9aea0547b9334314bd61be0f6cf90299/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:31.534770 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580d7e7-33a3-40ff-9398-2e6a64b96c00 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:31.536242 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-c580d7e7-33a3-40ff-9398-2e6a64b96c00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:31.611062 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c01133f-2ddd-4e2a-9d4d-0f459dd6ec21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:31.612961 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01133f-2ddd-4e2a-9d4d-0f459dd6ec21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:31.614223 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01133f-2ddd-4e2a-9d4d-0f459dd6ec21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:31.615292 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01133f-2ddd-4e2a-9d4d-0f459dd6ec21 admin admin] PATH_INFO: `/v3/projects/21fbb46ad3a949da935fd35b22bcc60f/users/152c1070e5964c469d2790f2cfe85236/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:31.617584 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c01133f-2ddd-4e2a-9d4d-0f459dd6ec21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21fbb46ad3a949da935fd35b22bcc60f, user_id=152c1070e5964c469d2790f2cfe85236, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:31.621051 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c01133f-2ddd-4e2a-9d4d-0f459dd6ec21 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:31.645824 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 869/1712] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:20:31 2022] PUT /identity/v3/projects/21fbb46ad3a949da935fd35b22bcc60f/users/152c1070e5964c469d2790f2cfe85236/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:31.648680 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c01133f-2ddd-4e2a-9d4d-0f459dd6ec21 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:31.661960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:31.688743 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580d7e7-33a3-40ff-9398-2e6a64b96c00 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:31.756400 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c01133f-2ddd-4e2a-9d4d-0f459dd6ec21 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:31.770471 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a45be-6021-4f8b-bfcd-7ddd8d492000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:31.771477 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a45be-6021-4f8b-bfcd-7ddd8d492000 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:31.772108 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a45be-6021-4f8b-bfcd-7ddd8d492000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:31.772628 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a45be-6021-4f8b-bfcd-7ddd8d492000 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:31.774377 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5a45be-6021-4f8b-bfcd-7ddd8d492000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:31.776362 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5a45be-6021-4f8b-bfcd-7ddd8d492000 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:31.791128 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 870/1713] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:31.792851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d449f0c-21b3-4b4c-8bb0-7eb52004003c None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:31.793251 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d449f0c-21b3-4b4c-8bb0-7eb52004003c None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:31.793988 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d449f0c-21b3-4b4c-8bb0-7eb52004003c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:31.806338 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f024aab-1efe-4cd0-a494-ceee4f76b853 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:31.807237 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f024aab-1efe-4cd0-a494-ceee4f76b853 service nova] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:31.807708 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f024aab-1efe-4cd0-a494-ceee4f76b853 service nova] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:31.808121 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f024aab-1efe-4cd0-a494-ceee4f76b853 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:31.826877 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-8d449f0c-21b3-4b4c-8bb0-7eb52004003c None None] MFA Rules not processed for user `2e8380a466484c83bca842518d171048`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:31.828476 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-8d449f0c-21b3-4b4c-8bb0-7eb52004003c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:31.870917 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:31.915713 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f024aab-1efe-4cd0-a494-ceee4f76b853 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:31.915713 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f024aab-1efe-4cd0-a494-ceee4f76b853 service nova] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:31.932626 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 844/1714] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:20:31.933479 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f024aab-1efe-4cd0-a494-ceee4f76b853 service nova] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:31.940999 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 871/1715] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:31 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:31.941740 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d449f0c-21b3-4b4c-8bb0-7eb52004003c None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:31.955568 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d449f0c-21b3-4b4c-8bb0-7eb52004003c None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:31.961588 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba925e8d-bc7f-4feb-ad5f-fa1c53e09d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:31.962440 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba925e8d-bc7f-4feb-ad5f-fa1c53e09d6d admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:31.962824 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba925e8d-bc7f-4feb-ad5f-fa1c53e09d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:31.963362 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba925e8d-bc7f-4feb-ad5f-fa1c53e09d6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:31.964159 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba925e8d-bc7f-4feb-ad5f-fa1c53e09d6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:31.965962 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba925e8d-bc7f-4feb-ad5f-fa1c53e09d6d admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:31.966158 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f024aab-1efe-4cd0-a494-ceee4f76b853 service nova] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:31.978082 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 872/1716] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:31.982792 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8fdb92-e3a2-4818-9931-aa3bd1996bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:31.982792 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fdb92-e3a2-4818-9931-aa3bd1996bdf admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:31.982792 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fdb92-e3a2-4818-9931-aa3bd1996bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:31.982792 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fdb92-e3a2-4818-9931-aa3bd1996bdf admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:31.983964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8fdb92-e3a2-4818-9931-aa3bd1996bdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:31.986208 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8fdb92-e3a2-4818-9931-aa3bd1996bdf admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:31.986434 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8845f-f5dc-4f90-9f5d-2288ba5a6fb1 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:31.986873 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8845f-f5dc-4f90-9f5d-2288ba5a6fb1 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:31.987577 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8845f-f5dc-4f90-9f5d-2288ba5a6fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:32.035223 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-3bf8845f-f5dc-4f90-9f5d-2288ba5a6fb1 None None] MFA Rules not processed for user `152c1070e5964c469d2790f2cfe85236`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:32.040590 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-3bf8845f-f5dc-4f90-9f5d-2288ba5a6fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:32.136671 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 845/1717] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:20:31 2022] POST /identity/v3/users => generated 294 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:32.137955 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8fdb92-e3a2-4818-9931-aa3bd1996bdf admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:32.159094 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8fdb92-e3a2-4818-9931-aa3bd1996bdf admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:32.184625 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92077d-7389-4a78-a937-fef75ba1324e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:32.185294 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92077d-7389-4a78-a937-fef75ba1324e admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:32.185584 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92077d-7389-4a78-a937-fef75ba1324e admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:32.185865 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92077d-7389-4a78-a937-fef75ba1324e admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:32.195724 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb92077d-7389-4a78-a937-fef75ba1324e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:32.199858 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb92077d-7389-4a78-a937-fef75ba1324e admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:32.204902 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:32.218774 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 846/1718] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:32.220020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92077d-7389-4a78-a937-fef75ba1324e admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:32.224554 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-eb92077d-7389-4a78-a937-fef75ba1324e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:32.241150 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 873/1719] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:31 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:32.242534 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8845f-f5dc-4f90-9f5d-2288ba5a6fb1 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:32.271128 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92077d-7389-4a78-a937-fef75ba1324e admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:32.297021 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8845f-f5dc-4f90-9f5d-2288ba5a6fb1 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:32.307768 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a2063-d5f3-4e52-a90e-9a5d64410db4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:32.308533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a2063-d5f3-4e52-a90e-9a5d64410db4 service nova] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:32.308828 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a2063-d5f3-4e52-a90e-9a5d64410db4 service nova] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:32.309110 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a2063-d5f3-4e52-a90e-9a5d64410db4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:32.320175 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e78af-1825-40d7-b9fe-784163209bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:32.321014 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e78af-1825-40d7-b9fe-784163209bbb admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:32.322074 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e78af-1825-40d7-b9fe-784163209bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:32.322074 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e78af-1825-40d7-b9fe-784163209bbb admin admin] PATH_INFO: `/v3/projects/1e4ad0eb37e4424d8fd4492518a88369/users/9aea0547b9334314bd61be0f6cf90299/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:32.325292 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e78af-1825-40d7-b9fe-784163209bbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e4ad0eb37e4424d8fd4492518a88369, user_id=9aea0547b9334314bd61be0f6cf90299, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:32.332574 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a2063-d5f3-4e52-a90e-9a5d64410db4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:32.337034 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a2063-d5f3-4e52-a90e-9a5d64410db4 service nova] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:32.339828 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e78af-1825-40d7-b9fe-784163209bbb admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:32.361255 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 847/1720] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:32 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:20:32.366981 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a2063-d5f3-4e52-a90e-9a5d64410db4 service nova] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:32.375628 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 874/1721] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:20:32 2022] PUT /identity/v3/projects/1e4ad0eb37e4424d8fd4492518a88369/users/9aea0547b9334314bd61be0f6cf90299/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:32.375628 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e78af-1825-40d7-b9fe-784163209bbb admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:32.389438 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a2063-d5f3-4e52-a90e-9a5d64410db4 service nova] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:32.457135 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e78af-1825-40d7-b9fe-784163209bbb admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:32.465428 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da0a2a-6b9d-438f-901b-4d537862b5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:32.466184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da0a2a-6b9d-438f-901b-4d537862b5fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:32.466771 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da0a2a-6b9d-438f-901b-4d537862b5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:32.467098 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da0a2a-6b9d-438f-901b-4d537862b5fb admin admin] PATH_INFO: `/v3/projects/21b90878aee347a08b8cc639839f4f46/users/dd57f2a3af68474d8ad1d602c2c55db6/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:32.485539 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da0a2a-6b9d-438f-901b-4d537862b5fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=21b90878aee347a08b8cc639839f4f46, user_id=dd57f2a3af68474d8ad1d602c2c55db6, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:32.492872 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da0a2a-6b9d-438f-901b-4d537862b5fb admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:32.504795 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60531a8a-bf9c-45e0-a00a-29cefd6e804a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:32.506221 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60531a8a-bf9c-45e0-a00a-29cefd6e804a admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:32.506616 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60531a8a-bf9c-45e0-a00a-29cefd6e804a admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:32.506900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60531a8a-bf9c-45e0-a00a-29cefd6e804a admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:32.507731 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60531a8a-bf9c-45e0-a00a-29cefd6e804a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:32.509884 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60531a8a-bf9c-45e0-a00a-29cefd6e804a admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:32.524512 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 848/1722] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:32.545888 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60531a8a-bf9c-45e0-a00a-29cefd6e804a admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:32.549036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-60531a8a-bf9c-45e0-a00a-29cefd6e804a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:32.603585 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 875/1723] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:20:32 2022] PUT /identity/v3/projects/21b90878aee347a08b8cc639839f4f46/users/dd57f2a3af68474d8ad1d602c2c55db6/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:32.606381 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370d680-c1cb-4174-afdd-66437c836121 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:32.606679 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370d680-c1cb-4174-afdd-66437c836121 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:32.606975 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370d680-c1cb-4174-afdd-66437c836121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:32.637896 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60531a8a-bf9c-45e0-a00a-29cefd6e804a admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:32.677540 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-8370d680-c1cb-4174-afdd-66437c836121 None None] MFA Rules not processed for user `a830bd2267f948f693c6b211293c6b89`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:32.678909 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-8370d680-c1cb-4174-afdd-66437c836121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:32.755201 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7a307-e820-4e85-af33-c40f3ec1701f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:32.756529 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7a307-e820-4e85-af33-c40f3ec1701f service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:32.756821 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7a307-e820-4e85-af33-c40f3ec1701f service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:32.757306 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7a307-e820-4e85-af33-c40f3ec1701f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:32.833489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 876/1724] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:32.834574 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81183ac-d8a9-408a-99b6-0cce584f1834 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:32.834865 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81183ac-d8a9-408a-99b6-0cce584f1834 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:32.835151 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81183ac-d8a9-408a-99b6-0cce584f1834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:32.906744 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd7a307-e820-4e85-af33-c40f3ec1701f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:32.908981 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd7a307-e820-4e85-af33-c40f3ec1701f service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:32.911860 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-a81183ac-d8a9-408a-99b6-0cce584f1834 None None] MFA Rules not processed for user `9aea0547b9334314bd61be0f6cf90299`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:32.913113 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-a81183ac-d8a9-408a-99b6-0cce584f1834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:32.922437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 849/1725] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:32 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:20:32.924519 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7a307-e820-4e85-af33-c40f3ec1701f service cinder] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:33.091095 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7a307-e820-4e85-af33-c40f3ec1701f service cinder] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:33.117394 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:33.118622 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b635f0f-4115-4f2a-bed3-8b2d62c70920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:33.119268 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b635f0f-4115-4f2a-bed3-8b2d62c70920 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:33.119580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b635f0f-4115-4f2a-bed3-8b2d62c70920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:33.125823 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b635f0f-4115-4f2a-bed3-8b2d62c70920 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:33.125823 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b635f0f-4115-4f2a-bed3-8b2d62c70920 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:33.125823 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b635f0f-4115-4f2a-bed3-8b2d62c70920 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:33.128727 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 877/1726] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:32 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:33.130792 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e50ea0-d24d-47d3-ae68-e76c0a1e485d None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:33.137095 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e50ea0-d24d-47d3-ae68-e76c0a1e485d None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:33.137389 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e50ea0-d24d-47d3-ae68-e76c0a1e485d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:33.160378 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 850/1727] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:33.161839 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de17c3-9939-4e8e-a277-55c4e54bd656 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:33.162132 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de17c3-9939-4e8e-a277-55c4e54bd656 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:33.162416 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de17c3-9939-4e8e-a277-55c4e54bd656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:33.208324 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-80de17c3-9939-4e8e-a277-55c4e54bd656 None None] MFA Rules not processed for user `a830bd2267f948f693c6b211293c6b89`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:33.209141 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-b0e50ea0-d24d-47d3-ae68-e76c0a1e485d None None] MFA Rules not processed for user `dd57f2a3af68474d8ad1d602c2c55db6`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:33.209716 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-80de17c3-9939-4e8e-a277-55c4e54bd656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:33.211596 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-b0e50ea0-d24d-47d3-ae68-e76c0a1e485d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:33.218883 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 851/1728] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:33.219951 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80de17c3-9939-4e8e-a277-55c4e54bd656 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:33.232670 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80de17c3-9939-4e8e-a277-55c4e54bd656 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:33.243737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e144db-33b4-4909-96a7-7f5f1c26e62a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:33.244689 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e144db-33b4-4909-96a7-7f5f1c26e62a service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:33.245236 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e144db-33b4-4909-96a7-7f5f1c26e62a service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:33.245611 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e144db-33b4-4909-96a7-7f5f1c26e62a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:33.260687 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e144db-33b4-4909-96a7-7f5f1c26e62a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:33.262417 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e144db-33b4-4909-96a7-7f5f1c26e62a service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:33.277196 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 852/1729] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:33 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:20:33.278276 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e144db-33b4-4909-96a7-7f5f1c26e62a service cinder] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:33.295304 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e144db-33b4-4909-96a7-7f5f1c26e62a service cinder] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:33.303945 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab503aa-7fdc-48af-ae71-fba0e7578e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:33.305540 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab503aa-7fdc-48af-ae71-fba0e7578e73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:33.306196 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab503aa-7fdc-48af-ae71-fba0e7578e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:33.306559 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab503aa-7fdc-48af-ae71-fba0e7578e73 admin admin] PATH_INFO: `/v3/projects/9464c94bc3ce422ea2d19ef4f2fa986e/users/cd542e8e5c604164a129b0239a43dec1/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:33.336437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab503aa-7fdc-48af-ae71-fba0e7578e73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9464c94bc3ce422ea2d19ef4f2fa986e, user_id=cd542e8e5c604164a129b0239a43dec1, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:33.342053 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 878/1730] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:33.344507 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc31cb-df83-4caf-8b08-071ce2adda18 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:33.345042 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc31cb-df83-4caf-8b08-071ce2adda18 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:33.345588 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc31cb-df83-4caf-8b08-071ce2adda18 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:33.346756 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab503aa-7fdc-48af-ae71-fba0e7578e73 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:33.353888 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 879/1731] 10.222.0.98 () {54 vars in 912 bytes} [Thu Aug 18 12:20:33 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:20:33.355197 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc31cb-df83-4caf-8b08-071ce2adda18 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:33.458735 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc31cb-df83-4caf-8b08-071ce2adda18 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:33.465931 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72bc3a2-e0c0-4627-8fd0-fa29d9de85c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:33.478570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72bc3a2-e0c0-4627-8fd0-fa29d9de85c7 service nova] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:33.478705 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 853/1732] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:20:33 2022] PUT /identity/v3/projects/9464c94bc3ce422ea2d19ef4f2fa986e/users/cd542e8e5c604164a129b0239a43dec1/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:33.478796 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72bc3a2-e0c0-4627-8fd0-fa29d9de85c7 service nova] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:33.478796 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72bc3a2-e0c0-4627-8fd0-fa29d9de85c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:33.479864 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab503aa-7fdc-48af-ae71-fba0e7578e73 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:33.482511 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-0ab503aa-7fdc-48af-ae71-fba0e7578e73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:33.490542 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72bc3a2-e0c0-4627-8fd0-fa29d9de85c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:33.492464 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72bc3a2-e0c0-4627-8fd0-fa29d9de85c7 service nova] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:33.517059 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 880/1733] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:33 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:20:33.518078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72bc3a2-e0c0-4627-8fd0-fa29d9de85c7 service nova] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:33.531646 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72bc3a2-e0c0-4627-8fd0-fa29d9de85c7 service nova] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:33.621320 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab503aa-7fdc-48af-ae71-fba0e7578e73 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:33.628942 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279fdb3-be99-4d9b-b154-c327826ea721 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:33.629696 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279fdb3-be99-4d9b-b154-c327826ea721 service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:33.629893 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279fdb3-be99-4d9b-b154-c327826ea721 service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:33.630170 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279fdb3-be99-4d9b-b154-c327826ea721 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:33.632015 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-e279fdb3-be99-4d9b-b154-c327826ea721 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:33.661909 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe36769-82c5-481a-bd65-d7b6d6e130d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:33.662664 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe36769-82c5-481a-bd65-d7b6d6e130d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:33.664125 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe36769-82c5-481a-bd65-d7b6d6e130d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:33.665314 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe36769-82c5-481a-bd65-d7b6d6e130d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:33.666053 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe36769-82c5-481a-bd65-d7b6d6e130d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:33.670316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe36769-82c5-481a-bd65-d7b6d6e130d6 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:33.697566 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:33.729238 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279fdb3-be99-4d9b-b154-c327826ea721 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:33.732668 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279fdb3-be99-4d9b-b154-c327826ea721 service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:33.742843 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 854/1734] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:20:33.744223 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4bd63-addd-4362-9f38-2eef6e152747 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:33.744428 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4bd63-addd-4362-9f38-2eef6e152747 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:33.744814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4bd63-addd-4362-9f38-2eef6e152747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:33.789497 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 881/1735] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:20:33 2022] POST /identity/v3/projects => generated 375 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:33.793682 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2741e9d-1ec8-4be8-adf5-5f87e9a08792 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:33.794135 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2741e9d-1ec8-4be8-adf5-5f87e9a08792 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:33.796591 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2741e9d-1ec8-4be8-adf5-5f87e9a08792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:33.808535 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-cff4bd63-addd-4362-9f38-2eef6e152747 None None] MFA Rules not processed for user `185644affb284c2b91e604cf99803ffb`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:33.810059 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-cff4bd63-addd-4362-9f38-2eef6e152747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:33.860401 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-c2741e9d-1ec8-4be8-adf5-5f87e9a08792 None None] MFA Rules not processed for user `cd542e8e5c604164a129b0239a43dec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:33.861674 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-c2741e9d-1ec8-4be8-adf5-5f87e9a08792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:33.875550 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 855/1736] 10.222.0.98 () {58 vars in 1009 bytes} [Thu Aug 18 12:20:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:33.876705 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4bd63-addd-4362-9f38-2eef6e152747 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:33.878232 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-cff4bd63-addd-4362-9f38-2eef6e152747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:33.967902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 882/1737] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:33 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:33.969484 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2741e9d-1ec8-4be8-adf5-5f87e9a08792 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:34.014350 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4bd63-addd-4362-9f38-2eef6e152747 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:34.027691 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff518c22-4205-4d51-b369-91c0f446130b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:34.028597 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff518c22-4205-4d51-b369-91c0f446130b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:34.028844 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff518c22-4205-4d51-b369-91c0f446130b admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:34.029264 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff518c22-4205-4d51-b369-91c0f446130b admin admin] PATH_INFO: `/v3/users/26b08395783d4d48916fae7f47ee79eb` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:34.032988 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff518c22-4205-4d51-b369-91c0f446130b admin admin] RBAC: Authorizing `identity:delete_user(user_id=26b08395783d4d48916fae7f47ee79eb)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:34.038988 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff518c22-4205-4d51-b369-91c0f446130b admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:34.112244 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2741e9d-1ec8-4be8-adf5-5f87e9a08792 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:34.130840 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c2a25-577c-4fa7-b715-e8bb69e842bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:34.131517 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c2a25-577c-4fa7-b715-e8bb69e842bf service cinder] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:34.131805 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c2a25-577c-4fa7-b715-e8bb69e842bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:34.132086 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c2a25-577c-4fa7-b715-e8bb69e842bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:34.134773 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-de0c2a25-577c-4fa7-b715-e8bb69e842bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:34.152171 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:34.212375 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0c2a25-577c-4fa7-b715-e8bb69e842bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:34.214307 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0c2a25-577c-4fa7-b715-e8bb69e842bf service cinder] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:34.230226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 883/1738] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:20:34.231336 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c2a25-577c-4fa7-b715-e8bb69e842bf service cinder] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:34.259136 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c2a25-577c-4fa7-b715-e8bb69e842bf service cinder] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:34.276512 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf03d69-ec98-464f-9ae8-bca09f55e27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:34.277263 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf03d69-ec98-464f-9ae8-bca09f55e27b admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:34.277730 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf03d69-ec98-464f-9ae8-bca09f55e27b admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:34.286177 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf03d69-ec98-464f-9ae8-bca09f55e27b admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:34.286177 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf03d69-ec98-464f-9ae8-bca09f55e27b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:34.286177 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf03d69-ec98-464f-9ae8-bca09f55e27b admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:34.293567 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-ff518c22-4205-4d51-b369-91c0f446130b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26b08395783d4d48916fae7f47ee79eb'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:34.295072 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-ff518c22-4205-4d51-b369-91c0f446130b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26b08395783d4d48916fae7f47ee79eb'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:34.299178 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-ff518c22-4205-4d51-b369-91c0f446130b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26b08395783d4d48916fae7f47ee79eb'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:34.328791 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-ff518c22-4205-4d51-b369-91c0f446130b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26b08395783d4d48916fae7f47ee79eb'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:34.355264 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-ff518c22-4205-4d51-b369-91c0f446130b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26b08395783d4d48916fae7f47ee79eb'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:34.403258 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 884/1739] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:20:34 2022] POST /identity/v3/users => generated 294 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:34.404679 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf03d69-ec98-464f-9ae8-bca09f55e27b admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:34.407558 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-fbf03d69-ec98-464f-9ae8-bca09f55e27b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:34.422173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf03d69-ec98-464f-9ae8-bca09f55e27b admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:34.428020 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df53f43-1988-4bd0-adcd-77f1193c6a1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:34.428729 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df53f43-1988-4bd0-adcd-77f1193c6a1c service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:34.429059 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df53f43-1988-4bd0-adcd-77f1193c6a1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:34.429587 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df53f43-1988-4bd0-adcd-77f1193c6a1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:34.431696 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-6df53f43-1988-4bd0-adcd-77f1193c6a1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:34.445828 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df53f43-1988-4bd0-adcd-77f1193c6a1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:34.452571 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df53f43-1988-4bd0-adcd-77f1193c6a1c service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:34.464816 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 885/1740] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:20:34.465848 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df53f43-1988-4bd0-adcd-77f1193c6a1c service neutron] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:34.468260 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-6df53f43-1988-4bd0-adcd-77f1193c6a1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:34.486158 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 856/1741] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:20:33 2022] DELETE /identity/v3/users/26b08395783d4d48916fae7f47ee79eb => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:34.487951 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8d58e-02d2-49f5-8f96-639f473f1554 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:34.488056 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df53f43-1988-4bd0-adcd-77f1193c6a1c service neutron] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:34.488300 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8d58e-02d2-49f5-8f96-639f473f1554 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:34.488702 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8d58e-02d2-49f5-8f96-639f473f1554 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:34.493749 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 857/1742] 10.222.0.98 () {54 vars in 912 bytes} [Thu Aug 18 12:20:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:20:34.495733 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8d58e-02d2-49f5-8f96-639f473f1554 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:34.503425 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-a2b8d58e-02d2-49f5-8f96-639f473f1554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:34.564878 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc0c9d8-1082-492f-baa8-6cb6a5c8ad66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:34.565575 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0c9d8-1082-492f-baa8-6cb6a5c8ad66 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:34.565866 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0c9d8-1082-492f-baa8-6cb6a5c8ad66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:34.566277 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0c9d8-1082-492f-baa8-6cb6a5c8ad66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:34.567019 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc0c9d8-1082-492f-baa8-6cb6a5c8ad66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:34.569515 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc0c9d8-1082-492f-baa8-6cb6a5c8ad66 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:34.662104 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8d58e-02d2-49f5-8f96-639f473f1554 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:34.680545 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc41683-e231-4cac-b5f0-f04433011949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:34.681195 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc41683-e231-4cac-b5f0-f04433011949 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:34.681477 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc41683-e231-4cac-b5f0-f04433011949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:34.681752 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc41683-e231-4cac-b5f0-f04433011949 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:34.685138 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc41683-e231-4cac-b5f0-f04433011949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:34.693959 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc41683-e231-4cac-b5f0-f04433011949 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:34.718754 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 858/1743] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:34.720586 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc7179-3995-42f8-b979-8b033b01cf37 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:34.722513 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc7179-3995-42f8-b979-8b033b01cf37 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:34.725148 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc7179-3995-42f8-b979-8b033b01cf37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:34.745633 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 886/1744] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:20:34 2022] POST /identity/v3/projects => generated 387 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:34.748778 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2345e-5aff-4f37-b7a0-25bc40991a83 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:34.750382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2345e-5aff-4f37-b7a0-25bc40991a83 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:34.750679 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2345e-5aff-4f37-b7a0-25bc40991a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:34.781593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-87d2345e-5aff-4f37-b7a0-25bc40991a83 None None] MFA Rules not processed for user `185644affb284c2b91e604cf99803ffb`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:34.783016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-87d2345e-5aff-4f37-b7a0-25bc40991a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:34.787516 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-c9bc7179-3995-42f8-b979-8b033b01cf37 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:34.788748 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-c9bc7179-3995-42f8-b979-8b033b01cf37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:34.861205 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:34.877650 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 887/1745] 10.222.0.98 () {58 vars in 1009 bytes} [Thu Aug 18 12:20:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:34.881131 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d2345e-5aff-4f37-b7a0-25bc40991a83 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:34.977029 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 859/1746] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:34.978420 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc7179-3995-42f8-b979-8b033b01cf37 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:34.990611 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc7179-3995-42f8-b979-8b033b01cf37 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:34.996630 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f494d2-afc1-41df-91d3-bd32ebecdf5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:34.997658 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f494d2-afc1-41df-91d3-bd32ebecdf5b admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:34.998399 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f494d2-afc1-41df-91d3-bd32ebecdf5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:34.998399 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f494d2-afc1-41df-91d3-bd32ebecdf5b admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:34.998914 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f494d2-afc1-41df-91d3-bd32ebecdf5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:35.002939 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f494d2-afc1-41df-91d3-bd32ebecdf5b admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:35.036208 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d2345e-5aff-4f37-b7a0-25bc40991a83 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:35.049978 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0fbd51-7a8b-4db5-b3de-35da5ceb6b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:35.050887 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0fbd51-7a8b-4db5-b3de-35da5ceb6b1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:35.051178 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0fbd51-7a8b-4db5-b3de-35da5ceb6b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:35.051704 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0fbd51-7a8b-4db5-b3de-35da5ceb6b1c admin admin] PATH_INFO: `/v3/projects/f4e8b88a3d7c44048186ed3302738ffc/users/bf4b6c4ff99c429d8a0d8459fe1877a0/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:35.071690 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0fbd51-7a8b-4db5-b3de-35da5ceb6b1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4e8b88a3d7c44048186ed3302738ffc, user_id=bf4b6c4ff99c429d8a0d8459fe1877a0, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:35.074414 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0fbd51-7a8b-4db5-b3de-35da5ceb6b1c admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:35.086549 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 860/1747] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:20:34 2022] POST /identity/v3/users => generated 300 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:35.087567 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f494d2-afc1-41df-91d3-bd32ebecdf5b admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:35.090197 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-18f494d2-afc1-41df-91d3-bd32ebecdf5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:35.142644 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f494d2-afc1-41df-91d3-bd32ebecdf5b admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:35.147042 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8f212-3add-4f79-b325-04d0956f4d71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:35.147804 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8f212-3add-4f79-b325-04d0956f4d71 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:35.148161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8f212-3add-4f79-b325-04d0956f4d71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:35.148513 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8f212-3add-4f79-b325-04d0956f4d71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:35.183381 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba8f212-3add-4f79-b325-04d0956f4d71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:35.184969 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba8f212-3add-4f79-b325-04d0956f4d71 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:35.187587 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:35.195257 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 861/1748] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:20:35.196360 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8f212-3add-4f79-b325-04d0956f4d71 service neutron] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:35.201858 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 888/1749] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:20:34 2022] PUT /identity/v3/projects/f4e8b88a3d7c44048186ed3302738ffc/users/bf4b6c4ff99c429d8a0d8459fe1877a0/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:35.204394 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0fbd51-7a8b-4db5-b3de-35da5ceb6b1c admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:35.220117 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8f212-3add-4f79-b325-04d0956f4d71 service neutron] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:35.304971 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4450e071-33fa-4572-959c-88d9c3b2b3a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:35.306071 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450e071-33fa-4572-959c-88d9c3b2b3a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:35.306681 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450e071-33fa-4572-959c-88d9c3b2b3a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:35.307177 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450e071-33fa-4572-959c-88d9c3b2b3a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:35.326219 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0fbd51-7a8b-4db5-b3de-35da5ceb6b1c admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:35.358963 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcca03d-f561-4240-a123-ff3147dd391c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:35.358963 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcca03d-f561-4240-a123-ff3147dd391c admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:35.358963 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcca03d-f561-4240-a123-ff3147dd391c admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:35.358963 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcca03d-f561-4240-a123-ff3147dd391c admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:35.360569 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcca03d-f561-4240-a123-ff3147dd391c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:35.360569 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcca03d-f561-4240-a123-ff3147dd391c admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:35.386944 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 889/1750] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:35.394064 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcca03d-f561-4240-a123-ff3147dd391c admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:35.459997 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcca03d-f561-4240-a123-ff3147dd391c admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:35.467368 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1590a566-9429-473a-8ed6-04a68085949f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:35.469091 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590a566-9429-473a-8ed6-04a68085949f admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:35.469631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590a566-9429-473a-8ed6-04a68085949f admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:35.470079 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590a566-9429-473a-8ed6-04a68085949f admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:35.473002 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1590a566-9429-473a-8ed6-04a68085949f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:35.474919 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1590a566-9429-473a-8ed6-04a68085949f admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:35.504476 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 890/1751] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:35.505870 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1590a566-9429-473a-8ed6-04a68085949f admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:35.506716 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4450e071-33fa-4572-959c-88d9c3b2b3a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:35.517692 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4450e071-33fa-4572-959c-88d9c3b2b3a1 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:35.527249 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1590a566-9429-473a-8ed6-04a68085949f admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:35.535746 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf69a563-6328-4b7e-8213-cd091383827b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:35.540700 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69a563-6328-4b7e-8213-cd091383827b admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:35.540700 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69a563-6328-4b7e-8213-cd091383827b admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:35.540700 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69a563-6328-4b7e-8213-cd091383827b admin admin] PATH_INFO: `/v3/projects/3a0ec913fb9f448f829fe3d35c57c7b4/users/fe354064fc954b1f8ee5d243303b9070/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:35.553059 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 862/1752] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:20:35.562483 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4450e071-33fa-4572-959c-88d9c3b2b3a1 service neutron] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:35.583555 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf69a563-6328-4b7e-8213-cd091383827b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a0ec913fb9f448f829fe3d35c57c7b4, user_id=fe354064fc954b1f8ee5d243303b9070, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:35.589811 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf69a563-6328-4b7e-8213-cd091383827b admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:35.602196 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4450e071-33fa-4572-959c-88d9c3b2b3a1 service neutron] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:35.618535 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b4a6e-15d7-4baf-8df1-12539d040632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:35.619225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b4a6e-15d7-4baf-8df1-12539d040632 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:35.619599 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b4a6e-15d7-4baf-8df1-12539d040632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:35.621011 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b4a6e-15d7-4baf-8df1-12539d040632 admin admin] PATH_INFO: `/v3/projects/f4e8b88a3d7c44048186ed3302738ffc/users/bf4b6c4ff99c429d8a0d8459fe1877a0/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:35.621921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302b4a6e-15d7-4baf-8df1-12539d040632 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4e8b88a3d7c44048186ed3302738ffc, user_id=bf4b6c4ff99c429d8a0d8459fe1877a0, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:35.625513 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302b4a6e-15d7-4baf-8df1-12539d040632 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:35.647627 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 863/1753] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:20:35 2022] PUT /identity/v3/projects/f4e8b88a3d7c44048186ed3302738ffc/users/bf4b6c4ff99c429d8a0d8459fe1877a0/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:35.657674 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751d965-0744-459e-996b-92dc15c238a8 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:35.657888 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751d965-0744-459e-996b-92dc15c238a8 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:35.658147 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751d965-0744-459e-996b-92dc15c238a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:35.680165 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 891/1754] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:20:35 2022] PUT /identity/v3/projects/3a0ec913fb9f448f829fe3d35c57c7b4/users/fe354064fc954b1f8ee5d243303b9070/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:35.685057 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf69a563-6328-4b7e-8213-cd091383827b admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:35.716842 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:35.725746 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-a751d965-0744-459e-996b-92dc15c238a8 None None] MFA Rules not processed for user `bf4b6c4ff99c429d8a0d8459fe1877a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:35.727184 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-a751d965-0744-459e-996b-92dc15c238a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:35.834203 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf69a563-6328-4b7e-8213-cd091383827b admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:35.849447 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec3867-97c1-4e53-9c86-072b7e4e4460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:35.853282 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec3867-97c1-4e53-9c86-072b7e4e4460 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:35.863578 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec3867-97c1-4e53-9c86-072b7e4e4460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:35.864153 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec3867-97c1-4e53-9c86-072b7e4e4460 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:35.868650 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec3867-97c1-4e53-9c86-072b7e4e4460 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:35.876506 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec3867-97c1-4e53-9c86-072b7e4e4460 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:35.927191 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 892/1755] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 12:20:35.935147 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd32d6b-e448-4ea7-8c83-caa0090f1278 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:35.935147 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd32d6b-e448-4ea7-8c83-caa0090f1278 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:35.937084 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd32d6b-e448-4ea7-8c83-caa0090f1278 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:35.944443 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 893/1756] 10.222.0.98 () {54 vars in 961 bytes} [Thu Aug 18 12:20:35 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:20:35.951365 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 864/1757] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:35 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:35.951365 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751d965-0744-459e-996b-92dc15c238a8 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:35.951622 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dbfe7e-58c8-4d2e-829a-c9c6429e61ec None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:35.951622 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dbfe7e-58c8-4d2e-829a-c9c6429e61ec None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:35.951622 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dbfe7e-58c8-4d2e-829a-c9c6429e61ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:35.984517 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751d965-0744-459e-996b-92dc15c238a8 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:35.989991 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b208f44-b5db-4a1d-bf17-376d7ade0673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:35.991376 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b208f44-b5db-4a1d-bf17-376d7ade0673 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:35.991376 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b208f44-b5db-4a1d-bf17-376d7ade0673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:35.991376 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b208f44-b5db-4a1d-bf17-376d7ade0673 admin admin] PATH_INFO: `/v3/projects/3a0ec913fb9f448f829fe3d35c57c7b4/users/fe354064fc954b1f8ee5d243303b9070/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:35.993269 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b208f44-b5db-4a1d-bf17-376d7ade0673 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a0ec913fb9f448f829fe3d35c57c7b4, user_id=fe354064fc954b1f8ee5d243303b9070, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:35.996696 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b208f44-b5db-4a1d-bf17-376d7ade0673 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:36.026118 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 865/1758] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:20:35 2022] PUT /identity/v3/projects/3a0ec913fb9f448f829fe3d35c57c7b4/users/fe354064fc954b1f8ee5d243303b9070/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:36.026118 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831d525-9327-42a7-94af-897b05d4673f None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:36.026118 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831d525-9327-42a7-94af-897b05d4673f None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:36.026118 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831d525-9327-42a7-94af-897b05d4673f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:36.032490 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-68dbfe7e-58c8-4d2e-829a-c9c6429e61ec None None] MFA Rules not processed for user `2e8380a466484c83bca842518d171048`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:36.044459 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-68dbfe7e-58c8-4d2e-829a-c9c6429e61ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:36.047331 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-d831d525-9327-42a7-94af-897b05d4673f None None] MFA Rules not processed for user `193637acadc14ec4a48ab79566ba3dab`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:36.055193 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-d831d525-9327-42a7-94af-897b05d4673f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:36.088083 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:36.155853 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 866/1759] 10.222.0.98 () {58 vars in 1058 bytes} [Thu Aug 18 12:20:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:36.156879 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831d525-9327-42a7-94af-897b05d4673f None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:36.158196 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-d831d525-9327-42a7-94af-897b05d4673f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:36.201569 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 894/1760] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:35 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:36.207618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5505ed-f1ae-4106-bd94-edc62837089f None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:36.208255 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5505ed-f1ae-4106-bd94-edc62837089f None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:36.210319 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5505ed-f1ae-4106-bd94-edc62837089f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:36.241395 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831d525-9327-42a7-94af-897b05d4673f None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:36.250606 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed88c246-73a2-4e54-8ce5-714866810b1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:36.250606 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88c246-73a2-4e54-8ce5-714866810b1e service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:36.250606 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88c246-73a2-4e54-8ce5-714866810b1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:36.253169 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88c246-73a2-4e54-8ce5-714866810b1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:36.255136 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-ed88c246-73a2-4e54-8ce5-714866810b1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:36.314757 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-bf5505ed-f1ae-4106-bd94-edc62837089f None None] MFA Rules not processed for user `fe354064fc954b1f8ee5d243303b9070`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:36.319506 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-bf5505ed-f1ae-4106-bd94-edc62837089f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:36.393263 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed88c246-73a2-4e54-8ce5-714866810b1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:36.394721 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed88c246-73a2-4e54-8ce5-714866810b1e service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:36.409408 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 867/1761] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:20:36.412312 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed88c246-73a2-4e54-8ce5-714866810b1e service cinder] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:36.486619 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed88c246-73a2-4e54-8ce5-714866810b1e service cinder] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:36.502859 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae0d396-cab7-49d1-be54-be242cd57f1b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193637acadc14ec4a48ab79566ba3dab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:36.503940 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0d396-cab7-49d1-be54-be242cd57f1b service placement] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:36.504432 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0d396-cab7-49d1-be54-be242cd57f1b service placement] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:36.504973 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0d396-cab7-49d1-be54-be242cd57f1b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:36.510888 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-fae0d396-cab7-49d1-be54-be242cd57f1b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:36.541874 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 895/1762] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:36 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:36.558904 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae0d396-cab7-49d1-be54-be242cd57f1b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:36.560638 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae0d396-cab7-49d1-be54-be242cd57f1b service placement] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:36.590937 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 868/1763] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:20:36.680859 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:36.750984 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213510f-67b4-459f-8bbe-696aeb1dd656 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:36.752392 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213510f-67b4-459f-8bbe-696aeb1dd656 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:36.752877 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213510f-67b4-459f-8bbe-696aeb1dd656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:36.795762 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-b213510f-67b4-459f-8bbe-696aeb1dd656 None None] MFA Rules not processed for user `b5113b1a77fd4b5da175b29b40d2b636`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:36.795762 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-b213510f-67b4-459f-8bbe-696aeb1dd656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:36.890649 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:37.000904 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 896/1764] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:37.279303 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae0d396-cab7-49d1-be54-be242cd57f1b service placement] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:37.280863 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-fae0d396-cab7-49d1-be54-be242cd57f1b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:37.437684 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:37.524233 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae0d396-cab7-49d1-be54-be242cd57f1b service placement] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:37.558685 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47287ae7-9bb9-4a5e-a7ad-603a173e8d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:37.559269 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47287ae7-9bb9-4a5e-a7ad-603a173e8d47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:37.559574 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47287ae7-9bb9-4a5e-a7ad-603a173e8d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:37.559848 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47287ae7-9bb9-4a5e-a7ad-603a173e8d47 admin admin] PATH_INFO: `/v3/projects/524406c27cf045a08ac45f2cb420e944` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:37.560997 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47287ae7-9bb9-4a5e-a7ad-603a173e8d47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=524406c27cf045a08ac45f2cb420e944)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:37.566818 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47287ae7-9bb9-4a5e-a7ad-603a173e8d47 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:37.572980 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b213510f-67b4-459f-8bbe-696aeb1dd656 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:37.588869 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b213510f-67b4-459f-8bbe-696aeb1dd656 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:37.597773 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45729e48-d5a6-423b-a1f8-66230f7558d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:37.599454 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45729e48-d5a6-423b-a1f8-66230f7558d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:37.599454 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45729e48-d5a6-423b-a1f8-66230f7558d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:37.599454 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45729e48-d5a6-423b-a1f8-66230f7558d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:37.621458 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45729e48-d5a6-423b-a1f8-66230f7558d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:37.623282 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45729e48-d5a6-423b-a1f8-66230f7558d7 service cinder] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:37.652149 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 897/1765] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:37 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:20:37.721229 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:37.803762 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-47287ae7-9bb9-4a5e-a7ad-603a173e8d47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '524406c27cf045a08ac45f2cb420e944', 'request_id': 'req-47287ae7-9bb9-4a5e-a7ad-603a173e8d47'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:37.829277 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-47287ae7-9bb9-4a5e-a7ad-603a173e8d47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '524406c27cf045a08ac45f2cb420e944', 'request_id': 'req-47287ae7-9bb9-4a5e-a7ad-603a173e8d47'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:37.934937 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-47287ae7-9bb9-4a5e-a7ad-603a173e8d47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '524406c27cf045a08ac45f2cb420e944', 'request_id': 'req-47287ae7-9bb9-4a5e-a7ad-603a173e8d47'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:37.958468 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-47287ae7-9bb9-4a5e-a7ad-603a173e8d47 admin admin] The token cache is being invalidate because project 524406c27cf045a08ac45f2cb420e944 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:20:37.958753 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-47287ae7-9bb9-4a5e-a7ad-603a173e8d47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:37.963628 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-47287ae7-9bb9-4a5e-a7ad-603a173e8d47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:37.966756 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 869/1766] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:20:37 2022] DELETE /identity/v3/projects/524406c27cf045a08ac45f2cb420e944 => generated 0 bytes in 688 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:37.972650 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57348487-b1c9-4c80-a2be-41707d79bac6 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:37.973007 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57348487-b1c9-4c80-a2be-41707d79bac6 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:37.973464 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57348487-b1c9-4c80-a2be-41707d79bac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:38.040052 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-57348487-b1c9-4c80-a2be-41707d79bac6 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:38.040052 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-57348487-b1c9-4c80-a2be-41707d79bac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:38.304413 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174d47b-7eba-42b3-8693-1b589a8579e0 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:38.304931 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174d47b-7eba-42b3-8693-1b589a8579e0 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:38.305359 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174d47b-7eba-42b3-8693-1b589a8579e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:38.313424 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 898/1767] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:38.324927 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57348487-b1c9-4c80-a2be-41707d79bac6 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:38.334644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-57348487-b1c9-4c80-a2be-41707d79bac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:38.350792 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-e174d47b-7eba-42b3-8693-1b589a8579e0 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:38.352143 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-e174d47b-7eba-42b3-8693-1b589a8579e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:38.366835 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 870/1768] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:20:38.371384 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e174d47b-7eba-42b3-8693-1b589a8579e0 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:38.379548 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57348487-b1c9-4c80-a2be-41707d79bac6 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:38.407538 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5b0501-6593-4d0c-b730-9d624c007751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:38.408406 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b0501-6593-4d0c-b730-9d624c007751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:38.411340 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b0501-6593-4d0c-b730-9d624c007751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:38.411340 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b0501-6593-4d0c-b730-9d624c007751 admin admin] PATH_INFO: `/v3/users/cd542e8e5c604164a129b0239a43dec1` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:38.418683 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e174d47b-7eba-42b3-8693-1b589a8579e0 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:38.425251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329efee-b972-4d2a-8317-5fb66470c6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:38.425952 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329efee-b972-4d2a-8317-5fb66470c6a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:38.426396 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329efee-b972-4d2a-8317-5fb66470c6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:38.426686 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329efee-b972-4d2a-8317-5fb66470c6a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:38.427358 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e329efee-b972-4d2a-8317-5fb66470c6a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:20:38.427736 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e329efee-b972-4d2a-8317-5fb66470c6a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:38.429926 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e329efee-b972-4d2a-8317-5fb66470c6a4 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:38.435683 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5b0501-6593-4d0c-b730-9d624c007751 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd542e8e5c604164a129b0239a43dec1)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:38.448789 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5b0501-6593-4d0c-b730-9d624c007751 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:38.492863 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 871/1769] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:20:38.497735 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329efee-b972-4d2a-8317-5fb66470c6a4 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:38.532333 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329efee-b972-4d2a-8317-5fb66470c6a4 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:38.541206 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3881985-b071-44a3-8903-84657041a584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:38.542414 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3881985-b071-44a3-8903-84657041a584 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:38.542414 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3881985-b071-44a3-8903-84657041a584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:38.542636 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3881985-b071-44a3-8903-84657041a584 admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:38.543193 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3881985-b071-44a3-8903-84657041a584 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:38.545176 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3881985-b071-44a3-8903-84657041a584 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:38.737969 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 872/1770] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:20:38 2022] POST /identity/v3/projects => generated 355 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:38.742156 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3881985-b071-44a3-8903-84657041a584 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:38.807124 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-ca5b0501-6593-4d0c-b730-9d624c007751 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd542e8e5c604164a129b0239a43dec1'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:38.851923 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-ca5b0501-6593-4d0c-b730-9d624c007751 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd542e8e5c604164a129b0239a43dec1'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:38.856051 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-ca5b0501-6593-4d0c-b730-9d624c007751 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd542e8e5c604164a129b0239a43dec1'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:38.935117 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-ca5b0501-6593-4d0c-b730-9d624c007751 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd542e8e5c604164a129b0239a43dec1'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:38.952515 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3881985-b071-44a3-8903-84657041a584 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:38.966810 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c876af4-9d16-4e5f-8ced-7dc38287e5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:38.968676 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c876af4-9d16-4e5f-8ced-7dc38287e5ca admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:38.972899 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c876af4-9d16-4e5f-8ced-7dc38287e5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:38.972899 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c876af4-9d16-4e5f-8ced-7dc38287e5ca admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:38.974484 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c876af4-9d16-4e5f-8ced-7dc38287e5ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:38.984279 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c876af4-9d16-4e5f-8ced-7dc38287e5ca admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:39.067509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-ca5b0501-6593-4d0c-b730-9d624c007751 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd542e8e5c604164a129b0239a43dec1'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:39.078166 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 899/1771] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:20:38 2022] DELETE /identity/v3/users/cd542e8e5c604164a129b0239a43dec1 => generated 0 bytes in 751 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:39.080504 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6750e-6589-4b7f-acc9-7160b554de63 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:39.080856 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6750e-6589-4b7f-acc9-7160b554de63 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:39.081149 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6750e-6589-4b7f-acc9-7160b554de63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:39.160304 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 873/1772] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:20:38 2022] POST /identity/v3/users => generated 278 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:20:39.184859 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c876af4-9d16-4e5f-8ced-7dc38287e5ca admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:39.193005 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-9c876af4-9d16-4e5f-8ced-7dc38287e5ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:39.196154 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-30c6750e-6589-4b7f-acc9-7160b554de63 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:39.205361 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-30c6750e-6589-4b7f-acc9-7160b554de63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:39.231888 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c876af4-9d16-4e5f-8ced-7dc38287e5ca admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:39.381909 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:39.430242 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 900/1773] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:39.448827 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c6750e-6589-4b7f-acc9-7160b554de63 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:39.450382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-30c6750e-6589-4b7f-acc9-7160b554de63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:39.465771 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b6206-8aa0-4e91-be23-2191394c02cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:39.483926 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b6206-8aa0-4e91-be23-2191394c02cf admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:39.484622 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b6206-8aa0-4e91-be23-2191394c02cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:39.485104 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b6206-8aa0-4e91-be23-2191394c02cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:39.486130 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085b6206-8aa0-4e91-be23-2191394c02cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:39.488328 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085b6206-8aa0-4e91-be23-2191394c02cf admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:39.519886 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 874/1774] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:39 2022] GET /identity/v3/roles => generated 1420 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:39.521398 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b6206-8aa0-4e91-be23-2191394c02cf admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:39.554381 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b6206-8aa0-4e91-be23-2191394c02cf admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:39.564781 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ca7180-c458-4481-baac-29959e7b3c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:39.565468 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca7180-c458-4481-baac-29959e7b3c48 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:39.565759 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca7180-c458-4481-baac-29959e7b3c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:39.566042 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca7180-c458-4481-baac-29959e7b3c48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:39.566699 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ca7180-c458-4481-baac-29959e7b3c48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:39.568787 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ca7180-c458-4481-baac-29959e7b3c48 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:39.587751 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 875/1775] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:39 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:39.591654 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ca7180-c458-4481-baac-29959e7b3c48 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:39.599464 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c6750e-6589-4b7f-acc9-7160b554de63 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:39.604516 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ca7180-c458-4481-baac-29959e7b3c48 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:39.608088 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c83a8e-1201-4abf-8fc6-9186f264d4ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:39.608905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c83a8e-1201-4abf-8fc6-9186f264d4ca service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:39.609314 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c83a8e-1201-4abf-8fc6-9186f264d4ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:39.609685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c83a8e-1201-4abf-8fc6-9186f264d4ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:39.622649 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e41bb0-1213-4d6f-b2a6-00b9e4083045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:39.622649 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e41bb0-1213-4d6f-b2a6-00b9e4083045 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:39.622649 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e41bb0-1213-4d6f-b2a6-00b9e4083045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:39.622649 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e41bb0-1213-4d6f-b2a6-00b9e4083045 admin admin] PATH_INFO: `/v3/projects/55524dfedf924164a1d57f8b236c2afd/users/fb3b39af31d8480bac0ee3a8cfe60b43/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:39.626439 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c83a8e-1201-4abf-8fc6-9186f264d4ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:39.628153 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c83a8e-1201-4abf-8fc6-9186f264d4ca service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:39.637902 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e41bb0-1213-4d6f-b2a6-00b9e4083045 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55524dfedf924164a1d57f8b236c2afd, user_id=fb3b39af31d8480bac0ee3a8cfe60b43, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:39.641476 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e41bb0-1213-4d6f-b2a6-00b9e4083045 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:39.659209 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 901/1776] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:20:39.793520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:39.804843 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 876/1777] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:20:39 2022] PUT /identity/v3/projects/55524dfedf924164a1d57f8b236c2afd/users/fb3b39af31d8480bac0ee3a8cfe60b43/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:20:39.817733 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5418e-0ba2-47b6-8e61-dbf8259c6075 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:39.818071 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5418e-0ba2-47b6-8e61-dbf8259c6075 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:39.819229 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5418e-0ba2-47b6-8e61-dbf8259c6075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:39.888566 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-35e5418e-0ba2-47b6-8e61-dbf8259c6075 None None] MFA Rules not processed for user `fb3b39af31d8480bac0ee3a8cfe60b43`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:39.890620 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-35e5418e-0ba2-47b6-8e61-dbf8259c6075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:40.002073 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 902/1778] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:39 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:40.006680 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e41bb0-1213-4d6f-b2a6-00b9e4083045 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:40.083739 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e41bb0-1213-4d6f-b2a6-00b9e4083045 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:40.088866 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d781c60-939c-458b-9928-2fa384df2dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:40.089651 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d781c60-939c-458b-9928-2fa384df2dd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:40.090036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d781c60-939c-458b-9928-2fa384df2dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:40.090421 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d781c60-939c-458b-9928-2fa384df2dd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:40.091160 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d781c60-939c-458b-9928-2fa384df2dd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:40.093412 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d781c60-939c-458b-9928-2fa384df2dd0 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:40.205989 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5418e-0ba2-47b6-8e61-dbf8259c6075 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:40.207025 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 877/1779] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:20:40 2022] POST /identity/v3/projects => generated 357 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:40.207592 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-35e5418e-0ba2-47b6-8e61-dbf8259c6075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:40.210639 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d781c60-939c-458b-9928-2fa384df2dd0 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:40.244463 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5418e-0ba2-47b6-8e61-dbf8259c6075 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:40.289833 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d781c60-939c-458b-9928-2fa384df2dd0 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:40.297019 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f7b2b7-39f5-4cc4-99e3-472b9257c35a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:40.297019 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f7b2b7-39f5-4cc4-99e3-472b9257c35a admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:40.297019 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f7b2b7-39f5-4cc4-99e3-472b9257c35a admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:40.297330 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f7b2b7-39f5-4cc4-99e3-472b9257c35a admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:40.297909 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f7b2b7-39f5-4cc4-99e3-472b9257c35a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:40.300007 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f7b2b7-39f5-4cc4-99e3-472b9257c35a admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:40.357844 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b77be33-228c-40af-9889-e1e4ff7dafdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:40.359295 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77be33-228c-40af-9889-e1e4ff7dafdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:40.359630 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77be33-228c-40af-9889-e1e4ff7dafdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:40.359916 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77be33-228c-40af-9889-e1e4ff7dafdb admin admin] PATH_INFO: `/v3/projects/9464c94bc3ce422ea2d19ef4f2fa986e` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:40.361245 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b77be33-228c-40af-9889-e1e4ff7dafdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9464c94bc3ce422ea2d19ef4f2fa986e)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:40.363619 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b77be33-228c-40af-9889-e1e4ff7dafdb admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:40.418605 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 878/1780] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:20:40 2022] POST /identity/v3/users => generated 285 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:20:40.432773 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f7b2b7-39f5-4cc4-99e3-472b9257c35a admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:40.443810 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f7b2b7-39f5-4cc4-99e3-472b9257c35a admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:40.449022 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ffafe0-0dfb-42c0-8479-8b1c9abfdd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:40.450027 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffafe0-0dfb-42c0-8479-8b1c9abfdd40 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:40.450376 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffafe0-0dfb-42c0-8479-8b1c9abfdd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:40.450722 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffafe0-0dfb-42c0-8479-8b1c9abfdd40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:40.451485 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ffafe0-0dfb-42c0-8479-8b1c9abfdd40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:40.453328 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ffafe0-0dfb-42c0-8479-8b1c9abfdd40 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:40.464651 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 879/1781] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:40.469300 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ffafe0-0dfb-42c0-8479-8b1c9abfdd40 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:40.480157 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ffafe0-0dfb-42c0-8479-8b1c9abfdd40 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:40.485559 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dd4d03-bd8f-4e74-bafe-de01f1e365e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:40.486221 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd4d03-bd8f-4e74-bafe-de01f1e365e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:40.486504 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd4d03-bd8f-4e74-bafe-de01f1e365e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:40.486778 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd4d03-bd8f-4e74-bafe-de01f1e365e8 admin admin] PATH_INFO: `/v3/projects/939a72f1a4e74a36ba779d402750df84/users/b832ab1f97d44060b5eb31765fd59c75/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:40.502064 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dd4d03-bd8f-4e74-bafe-de01f1e365e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=939a72f1a4e74a36ba779d402750df84, user_id=b832ab1f97d44060b5eb31765fd59c75, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:40.504845 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dd4d03-bd8f-4e74-bafe-de01f1e365e8 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:40.620848 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 880/1782] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:20:40 2022] PUT /identity/v3/projects/939a72f1a4e74a36ba779d402750df84/users/b832ab1f97d44060b5eb31765fd59c75/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:20:40.624925 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dd4d03-bd8f-4e74-bafe-de01f1e365e8 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:40.631323 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-7b77be33-228c-40af-9889-e1e4ff7dafdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9464c94bc3ce422ea2d19ef4f2fa986e', 'request_id': 'req-7b77be33-228c-40af-9889-e1e4ff7dafdb'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:40.638095 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-7b77be33-228c-40af-9889-e1e4ff7dafdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9464c94bc3ce422ea2d19ef4f2fa986e', 'request_id': 'req-7b77be33-228c-40af-9889-e1e4ff7dafdb'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:40.700765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dd4d03-bd8f-4e74-bafe-de01f1e365e8 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:40.706328 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-7b77be33-228c-40af-9889-e1e4ff7dafdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9464c94bc3ce422ea2d19ef4f2fa986e', 'request_id': 'req-7b77be33-228c-40af-9889-e1e4ff7dafdb'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:40.725771 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-7b77be33-228c-40af-9889-e1e4ff7dafdb admin admin] The token cache is being invalidate because project 9464c94bc3ce422ea2d19ef4f2fa986e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:20:40.726088 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-7b77be33-228c-40af-9889-e1e4ff7dafdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:40.726852 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-7b77be33-228c-40af-9889-e1e4ff7dafdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:40.730481 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 903/1783] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:20:40 2022] DELETE /identity/v3/projects/9464c94bc3ce422ea2d19ef4f2fa986e => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:40.735470 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b77be33-228c-40af-9889-e1e4ff7dafdb admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:40.737116 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-7b77be33-228c-40af-9889-e1e4ff7dafdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:40.784906 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb2ebcb-2f49-43a6-ba64-cd49e6aa1a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:40.786214 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb2ebcb-2f49-43a6-ba64-cd49e6aa1a82 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:40.786609 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb2ebcb-2f49-43a6-ba64-cd49e6aa1a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:40.786985 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb2ebcb-2f49-43a6-ba64-cd49e6aa1a82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:40.787604 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb2ebcb-2f49-43a6-ba64-cd49e6aa1a82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:40.789294 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb2ebcb-2f49-43a6-ba64-cd49e6aa1a82 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:40.812986 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 881/1784] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 12:20:40.814041 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb2ebcb-2f49-43a6-ba64-cd49e6aa1a82 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:40.816513 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-ffb2ebcb-2f49-43a6-ba64-cd49e6aa1a82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:40.828430 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b77be33-228c-40af-9889-e1e4ff7dafdb admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:40.829979 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb2ebcb-2f49-43a6-ba64-cd49e6aa1a82 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:40.836240 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04999e72-c47c-484e-b85d-a43e62399df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:40.837134 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04999e72-c47c-484e-b85d-a43e62399df6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:40.837869 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04999e72-c47c-484e-b85d-a43e62399df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:40.838160 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04999e72-c47c-484e-b85d-a43e62399df6 admin admin] PATH_INFO: `/v3/users/fe354064fc954b1f8ee5d243303b9070` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:40.838559 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e3e992-0a6b-4a3f-a7c6-0bf159977527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:40.839290 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3e992-0a6b-4a3f-a7c6-0bf159977527 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:40.839530 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04999e72-c47c-484e-b85d-a43e62399df6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe354064fc954b1f8ee5d243303b9070)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:40.845290 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3e992-0a6b-4a3f-a7c6-0bf159977527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:40.845290 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3e992-0a6b-4a3f-a7c6-0bf159977527 admin admin] PATH_INFO: `/v3/projects/939a72f1a4e74a36ba779d402750df84/users/b832ab1f97d44060b5eb31765fd59c75/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:40.845290 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e3e992-0a6b-4a3f-a7c6-0bf159977527 admin admin] RBAC: Authorizing `identity:create_grant(project_id=939a72f1a4e74a36ba779d402750df84, user_id=b832ab1f97d44060b5eb31765fd59c75, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:40.845744 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04999e72-c47c-484e-b85d-a43e62399df6 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:40.845821 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e3e992-0a6b-4a3f-a7c6-0bf159977527 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:40.860317 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 882/1785] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:20:40 2022] PUT /identity/v3/projects/939a72f1a4e74a36ba779d402750df84/users/b832ab1f97d44060b5eb31765fd59c75/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:40.872150 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5721e2-82f8-47be-8a61-98bf30dd6987 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:40.872591 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5721e2-82f8-47be-8a61-98bf30dd6987 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:40.873022 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5721e2-82f8-47be-8a61-98bf30dd6987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:40.924217 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-4e5721e2-82f8-47be-8a61-98bf30dd6987 None None] MFA Rules not processed for user `b832ab1f97d44060b5eb31765fd59c75`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:40.925527 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-4e5721e2-82f8-47be-8a61-98bf30dd6987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:40.991176 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:40.996634 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 883/1786] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:40 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:41.009396 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5721e2-82f8-47be-8a61-98bf30dd6987 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:41.010850 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-4e5721e2-82f8-47be-8a61-98bf30dd6987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:41.047368 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-04999e72-c47c-484e-b85d-a43e62399df6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe354064fc954b1f8ee5d243303b9070'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:41.055212 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-04999e72-c47c-484e-b85d-a43e62399df6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe354064fc954b1f8ee5d243303b9070'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:41.056625 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-04999e72-c47c-484e-b85d-a43e62399df6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe354064fc954b1f8ee5d243303b9070'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:41.072526 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-04999e72-c47c-484e-b85d-a43e62399df6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe354064fc954b1f8ee5d243303b9070'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:41.080389 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5721e2-82f8-47be-8a61-98bf30dd6987 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:41.084828 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424def0-1e3d-41a2-a113-dcf896427570 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:41.085666 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424def0-1e3d-41a2-a113-dcf896427570 service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:41.086127 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424def0-1e3d-41a2-a113-dcf896427570 service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:41.086548 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424def0-1e3d-41a2-a113-dcf896427570 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:41.097084 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d424def0-1e3d-41a2-a113-dcf896427570 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:41.098540 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d424def0-1e3d-41a2-a113-dcf896427570 service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:41.111933 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 884/1787] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:41 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:20:41.164474 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-04999e72-c47c-484e-b85d-a43e62399df6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe354064fc954b1f8ee5d243303b9070'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:41.176027 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 904/1788] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:20:40 2022] DELETE /identity/v3/users/fe354064fc954b1f8ee5d243303b9070 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:20:41.361320 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424def0-1e3d-41a2-a113-dcf896427570 service cinder] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:41.370215 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424def0-1e3d-41a2-a113-dcf896427570 service cinder] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:41.428472 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc20574-be3d-43e1-9e72-c6a84ff38a52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:41.429107 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc20574-be3d-43e1-9e72-c6a84ff38a52 service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:41.429389 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc20574-be3d-43e1-9e72-c6a84ff38a52 service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:41.429817 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc20574-be3d-43e1-9e72-c6a84ff38a52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:41.431966 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-2bc20574-be3d-43e1-9e72-c6a84ff38a52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:41.484378 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:41.524801 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc20574-be3d-43e1-9e72-c6a84ff38a52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:41.526232 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc20574-be3d-43e1-9e72-c6a84ff38a52 service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:41.536949 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 885/1789] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:41 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:20:41.643628 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:41.725738 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04999e72-c47c-484e-b85d-a43e62399df6 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:41.727201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-04999e72-c47c-484e-b85d-a43e62399df6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:41.796531 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04999e72-c47c-484e-b85d-a43e62399df6 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:41.801988 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f82dd1-1c08-491c-a4ed-314724f2e186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:41.802648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f82dd1-1c08-491c-a4ed-314724f2e186 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:41.802933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f82dd1-1c08-491c-a4ed-314724f2e186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:41.803330 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f82dd1-1c08-491c-a4ed-314724f2e186 admin admin] PATH_INFO: `/v3/projects/3a0ec913fb9f448f829fe3d35c57c7b4` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:41.804425 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f82dd1-1c08-491c-a4ed-314724f2e186 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a0ec913fb9f448f829fe3d35c57c7b4)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:41.806456 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f82dd1-1c08-491c-a4ed-314724f2e186 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:41.918614 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-78f82dd1-1c08-491c-a4ed-314724f2e186 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a0ec913fb9f448f829fe3d35c57c7b4', 'request_id': 'req-78f82dd1-1c08-491c-a4ed-314724f2e186'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:41.923925 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:41.926560 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-78f82dd1-1c08-491c-a4ed-314724f2e186 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a0ec913fb9f448f829fe3d35c57c7b4', 'request_id': 'req-78f82dd1-1c08-491c-a4ed-314724f2e186'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:42.043914 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-78f82dd1-1c08-491c-a4ed-314724f2e186 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a0ec913fb9f448f829fe3d35c57c7b4', 'request_id': 'req-78f82dd1-1c08-491c-a4ed-314724f2e186'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:42.044203 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:42.057739 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-78f82dd1-1c08-491c-a4ed-314724f2e186 admin admin] The token cache is being invalidate because project 3a0ec913fb9f448f829fe3d35c57c7b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:20:42.058035 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-78f82dd1-1c08-491c-a4ed-314724f2e186 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:42.058702 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-78f82dd1-1c08-491c-a4ed-314724f2e186 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:42.060566 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 905/1790] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:20:41 2022] DELETE /identity/v3/projects/3a0ec913fb9f448f829fe3d35c57c7b4 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:42.065743 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6cc593-f22c-493a-9706-f995cbf42b3e None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:42.066036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6cc593-f22c-493a-9706-f995cbf42b3e None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:42.066317 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6cc593-f22c-493a-9706-f995cbf42b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:42.087151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-fc6cc593-f22c-493a-9706-f995cbf42b3e None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:42.088422 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-fc6cc593-f22c-493a-9706-f995cbf42b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:42.150959 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 886/1791] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:42.162512 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b9e5f-0250-4fdb-a311-061e7cef376e None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:42.162803 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b9e5f-0250-4fdb-a311-061e7cef376e None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:42.163100 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b9e5f-0250-4fdb-a311-061e7cef376e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:42.188271 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:42.192556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-608b9e5f-0250-4fdb-a311-061e7cef376e None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:42.193800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-608b9e5f-0250-4fdb-a311-061e7cef376e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:42.203430 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 906/1792] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:20:42.207712 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6cc593-f22c-493a-9706-f995cbf42b3e None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:42.217660 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6cc593-f22c-493a-9706-f995cbf42b3e None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:42.222432 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd095caf-d14b-49f2-a90d-08474f10e369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:42.223040 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd095caf-d14b-49f2-a90d-08474f10e369 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:42.223322 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd095caf-d14b-49f2-a90d-08474f10e369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:42.223626 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd095caf-d14b-49f2-a90d-08474f10e369 admin admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:42.224311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd095caf-d14b-49f2-a90d-08474f10e369 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:20:42.224618 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd095caf-d14b-49f2-a90d-08474f10e369 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:42.226206 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd095caf-d14b-49f2-a90d-08474f10e369 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:42.238575 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 887/1793] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:20:42.243249 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608b9e5f-0250-4fdb-a311-061e7cef376e None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:42.253472 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608b9e5f-0250-4fdb-a311-061e7cef376e None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:42.258193 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75d15b0-0755-459a-a045-6033c4d6ea24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:42.270449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d15b0-0755-459a-a045-6033c4d6ea24 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:42.270449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d15b0-0755-459a-a045-6033c4d6ea24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:42.270449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d15b0-0755-459a-a045-6033c4d6ea24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:42.270449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75d15b0-0755-459a-a045-6033c4d6ea24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:42.270449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75d15b0-0755-459a-a045-6033c4d6ea24 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:42.356425 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 907/1794] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:20:42 2022] POST /identity/v3/projects => generated 401 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:42.358805 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd095caf-d14b-49f2-a90d-08474f10e369 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:42.369745 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd095caf-d14b-49f2-a90d-08474f10e369 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:42.431759 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb66d50-5b33-40ad-bec8-2d302dd3b276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:42.432383 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb66d50-5b33-40ad-bec8-2d302dd3b276 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:42.432674 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb66d50-5b33-40ad-bec8-2d302dd3b276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:42.432987 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb66d50-5b33-40ad-bec8-2d302dd3b276 admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:42.433634 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb66d50-5b33-40ad-bec8-2d302dd3b276 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:42.435682 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb66d50-5b33-40ad-bec8-2d302dd3b276 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:42.562424 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 888/1795] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:20:42 2022] POST /identity/v3/users => generated 301 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:42.563498 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75d15b0-0755-459a-a045-6033c4d6ea24 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:42.574449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75d15b0-0755-459a-a045-6033c4d6ea24 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:42.580056 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb25f6-162b-4985-8319-7cf85ada43b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:42.580835 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb25f6-162b-4985-8319-7cf85ada43b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:42.581120 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb25f6-162b-4985-8319-7cf85ada43b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:42.581445 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb25f6-162b-4985-8319-7cf85ada43b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:42.582104 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb25f6-162b-4985-8319-7cf85ada43b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:42.585665 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb25f6-162b-4985-8319-7cf85ada43b1 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:42.598724 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 908/1796] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 12:20:42.602956 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb66d50-5b33-40ad-bec8-2d302dd3b276 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:42.614291 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb66d50-5b33-40ad-bec8-2d302dd3b276 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:42.623151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c5ef9f-afe0-482f-a5a5-fdc5f7fe637f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:42.623151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5ef9f-afe0-482f-a5a5-fdc5f7fe637f admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:42.623151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5ef9f-afe0-482f-a5a5-fdc5f7fe637f admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:42.623151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5ef9f-afe0-482f-a5a5-fdc5f7fe637f admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:42.623581 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c5ef9f-afe0-482f-a5a5-fdc5f7fe637f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:42.623581 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c5ef9f-afe0-482f-a5a5-fdc5f7fe637f admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:42.639584 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 889/1797] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:42.640908 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb25f6-162b-4985-8319-7cf85ada43b1 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:42.652570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb25f6-162b-4985-8319-7cf85ada43b1 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:42.658454 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771df00-b819-4299-9f8c-98958943a579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:42.659184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771df00-b819-4299-9f8c-98958943a579 admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:42.659841 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771df00-b819-4299-9f8c-98958943a579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:42.660490 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771df00-b819-4299-9f8c-98958943a579 admin admin] PATH_INFO: `/v3/projects/5c57cdf8e2c14790b249a67d360deeb4/users/12c8289effa743009a910410a3d0facd/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:42.662738 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:42.692372 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c771df00-b819-4299-9f8c-98958943a579 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c57cdf8e2c14790b249a67d360deeb4, user_id=12c8289effa743009a910410a3d0facd, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:42.695914 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c771df00-b819-4299-9f8c-98958943a579 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:42.784820 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 909/1798] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:20:42 2022] PUT /identity/v3/projects/5c57cdf8e2c14790b249a67d360deeb4/users/12c8289effa743009a910410a3d0facd/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:42.798537 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92b347-dc39-4f41-b50c-9bb38f8bf030 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:42.798839 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92b347-dc39-4f41-b50c-9bb38f8bf030 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:42.799094 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92b347-dc39-4f41-b50c-9bb38f8bf030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:42.838230 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-1b92b347-dc39-4f41-b50c-9bb38f8bf030 None None] MFA Rules not processed for user `12c8289effa743009a910410a3d0facd`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:42.839639 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-1b92b347-dc39-4f41-b50c-9bb38f8bf030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:42.912207 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 890/1799] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:42 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:42.916642 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771df00-b819-4299-9f8c-98958943a579 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:42.957864 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:42.984129 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771df00-b819-4299-9f8c-98958943a579 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:42.989302 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a9769-5770-4b26-99d4-adb5c6e4a371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:42.989918 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a9769-5770-4b26-99d4-adb5c6e4a371 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:42.990203 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a9769-5770-4b26-99d4-adb5c6e4a371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:42.996676 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a9769-5770-4b26-99d4-adb5c6e4a371 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:42.996676 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590a9769-5770-4b26-99d4-adb5c6e4a371 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:42.996676 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590a9769-5770-4b26-99d4-adb5c6e4a371 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:43.074189 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:43.082877 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 910/1800] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:20:42 2022] POST /identity/v3/projects => generated 399 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:43.087370 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92b347-dc39-4f41-b50c-9bb38f8bf030 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:43.104749 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92b347-dc39-4f41-b50c-9bb38f8bf030 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:43.140920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:43.176194 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d96fc-4b73-4b5a-9615-f58849fe7465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:43.179455 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d96fc-4b73-4b5a-9615-f58849fe7465 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:43.179455 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d96fc-4b73-4b5a-9615-f58849fe7465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:43.179455 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d96fc-4b73-4b5a-9615-f58849fe7465 admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:43.179455 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2d96fc-4b73-4b5a-9615-f58849fe7465 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:43.180376 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2d96fc-4b73-4b5a-9615-f58849fe7465 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:43.289742 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 891/1801] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:20:43 2022] POST /identity/v3/users => generated 306 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:43.294125 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a9769-5770-4b26-99d4-adb5c6e4a371 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:43.304437 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a9769-5770-4b26-99d4-adb5c6e4a371 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:43.309084 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0102d500-003b-4584-96f5-7e6d91f1ff21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:43.309696 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102d500-003b-4584-96f5-7e6d91f1ff21 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:43.309978 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102d500-003b-4584-96f5-7e6d91f1ff21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:43.310255 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102d500-003b-4584-96f5-7e6d91f1ff21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:43.317696 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0102d500-003b-4584-96f5-7e6d91f1ff21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:43.317696 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0102d500-003b-4584-96f5-7e6d91f1ff21 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:43.324804 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 911/1802] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:43.329809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d96fc-4b73-4b5a-9615-f58849fe7465 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:43.339834 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d96fc-4b73-4b5a-9615-f58849fe7465 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:43.344393 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04178c5-ad06-4102-bae5-96dc19b809c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:43.345033 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04178c5-ad06-4102-bae5-96dc19b809c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:43.345312 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04178c5-ad06-4102-bae5-96dc19b809c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:43.345605 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04178c5-ad06-4102-bae5-96dc19b809c0 admin admin] PATH_INFO: `/v3/projects/e0b3186e766d48dfa5272547c48d9a55/users/b7f41e5aac4f466a93003b9238746f9b/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:43.360822 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04178c5-ad06-4102-bae5-96dc19b809c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0b3186e766d48dfa5272547c48d9a55, user_id=b7f41e5aac4f466a93003b9238746f9b, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:43.363502 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04178c5-ad06-4102-bae5-96dc19b809c0 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:43.432802 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 892/1803] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:20:43 2022] PUT /identity/v3/projects/e0b3186e766d48dfa5272547c48d9a55/users/b7f41e5aac4f466a93003b9238746f9b/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:43.437431 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0102d500-003b-4584-96f5-7e6d91f1ff21 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:43.450496 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0102d500-003b-4584-96f5-7e6d91f1ff21 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:43.517279 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068a6590-a7f7-44a2-b5e3-2b2cb8e9f011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:43.518187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068a6590-a7f7-44a2-b5e3-2b2cb8e9f011 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:43.518488 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068a6590-a7f7-44a2-b5e3-2b2cb8e9f011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:43.518850 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068a6590-a7f7-44a2-b5e3-2b2cb8e9f011 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:43.519557 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068a6590-a7f7-44a2-b5e3-2b2cb8e9f011 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:43.521237 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068a6590-a7f7-44a2-b5e3-2b2cb8e9f011 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:43.533036 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 912/1804] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:43.539552 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04178c5-ad06-4102-bae5-96dc19b809c0 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:43.551374 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04178c5-ad06-4102-bae5-96dc19b809c0 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:43.556983 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb0670-9d76-4cb5-bd23-a7a8614023f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:43.557672 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb0670-9d76-4cb5-bd23-a7a8614023f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:43.557908 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb0670-9d76-4cb5-bd23-a7a8614023f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:43.558178 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb0670-9d76-4cb5-bd23-a7a8614023f2 admin admin] PATH_INFO: `/v3/projects/e0b3186e766d48dfa5272547c48d9a55/users/b7f41e5aac4f466a93003b9238746f9b/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:43.560198 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eb0670-9d76-4cb5-bd23-a7a8614023f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0b3186e766d48dfa5272547c48d9a55, user_id=b7f41e5aac4f466a93003b9238746f9b, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:43.563846 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eb0670-9d76-4cb5-bd23-a7a8614023f2 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:43.576250 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 893/1805] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:20:43 2022] PUT /identity/v3/projects/e0b3186e766d48dfa5272547c48d9a55/users/b7f41e5aac4f466a93003b9238746f9b/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:43.587324 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c221b72-6b5f-44f3-a128-27b4b6d57949 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:43.587646 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c221b72-6b5f-44f3-a128-27b4b6d57949 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:43.587932 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c221b72-6b5f-44f3-a128-27b4b6d57949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:43.615904 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-7c221b72-6b5f-44f3-a128-27b4b6d57949 None None] MFA Rules not processed for user `b7f41e5aac4f466a93003b9238746f9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:43.617122 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-7c221b72-6b5f-44f3-a128-27b4b6d57949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:43.678321 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:43.685603 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 913/1806] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:43 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:43.697725 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb0670-9d76-4cb5-bd23-a7a8614023f2 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:43.699673 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-01eb0670-9d76-4cb5-bd23-a7a8614023f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:43.759618 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb0670-9d76-4cb5-bd23-a7a8614023f2 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:43.763361 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995528b-1b58-42df-b17a-56408afa0d34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:43.764045 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995528b-1b58-42df-b17a-56408afa0d34 service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:43.764329 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995528b-1b58-42df-b17a-56408afa0d34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:43.764605 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995528b-1b58-42df-b17a-56408afa0d34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:43.770972 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:43.786399 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7995528b-1b58-42df-b17a-56408afa0d34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:43.788615 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7995528b-1b58-42df-b17a-56408afa0d34 service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:43.810632 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 894/1807] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:43 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:20:43.843520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:44.023000 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c221b72-6b5f-44f3-a128-27b4b6d57949 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:44.032739 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c221b72-6b5f-44f3-a128-27b4b6d57949 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:44.036494 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88e8dca-7dea-436d-b7f9-fe620f89aefd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:44.037114 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e8dca-7dea-436d-b7f9-fe620f89aefd service cinder] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:44.037404 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e8dca-7dea-436d-b7f9-fe620f89aefd service cinder] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:44.037681 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e8dca-7dea-436d-b7f9-fe620f89aefd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:44.095340 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e8dca-7dea-436d-b7f9-fe620f89aefd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:44.096816 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e8dca-7dea-436d-b7f9-fe620f89aefd service cinder] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:44.105978 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 914/1808] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:44 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:20:46.907208 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995528b-1b58-42df-b17a-56408afa0d34 service cinder] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:46.909333 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-7995528b-1b58-42df-b17a-56408afa0d34 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:47.075226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995528b-1b58-42df-b17a-56408afa0d34 service cinder] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:47.081569 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23242191-0a67-40d2-ad3a-e9bd4c3c931a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:47.082611 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23242191-0a67-40d2-ad3a-e9bd4c3c931a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:47.083700 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23242191-0a67-40d2-ad3a-e9bd4c3c931a admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:47.084165 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23242191-0a67-40d2-ad3a-e9bd4c3c931a admin admin] PATH_INFO: `/v3/users/dd57f2a3af68474d8ad1d602c2c55db6` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:47.087754 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23242191-0a67-40d2-ad3a-e9bd4c3c931a admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd57f2a3af68474d8ad1d602c2c55db6)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:47.093140 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23242191-0a67-40d2-ad3a-e9bd4c3c931a admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:47.334212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:47.357370 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-23242191-0a67-40d2-ad3a-e9bd4c3c931a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd57f2a3af68474d8ad1d602c2c55db6'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:47.358145 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-23242191-0a67-40d2-ad3a-e9bd4c3c931a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd57f2a3af68474d8ad1d602c2c55db6'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:47.358792 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-23242191-0a67-40d2-ad3a-e9bd4c3c931a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd57f2a3af68474d8ad1d602c2c55db6'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:47.375706 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-23242191-0a67-40d2-ad3a-e9bd4c3c931a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd57f2a3af68474d8ad1d602c2c55db6'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:47.397354 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-23242191-0a67-40d2-ad3a-e9bd4c3c931a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd57f2a3af68474d8ad1d602c2c55db6'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:47.470578 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 895/1809] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:20:46 2022] DELETE /identity/v3/users/dd57f2a3af68474d8ad1d602c2c55db6 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:47.476007 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda8121-d8e5-4f7d-8d1b-1979ae1c15f0 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:47.476988 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda8121-d8e5-4f7d-8d1b-1979ae1c15f0 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:47.477368 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda8121-d8e5-4f7d-8d1b-1979ae1c15f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:47.550624 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a52a5a6-691c-4bb3-8134-0d1b235ecd74 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:47.550906 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a52a5a6-691c-4bb3-8134-0d1b235ecd74 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:47.551186 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a52a5a6-691c-4bb3-8134-0d1b235ecd74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:47.554969 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-abda8121-d8e5-4f7d-8d1b-1979ae1c15f0 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:47.559164 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-abda8121-d8e5-4f7d-8d1b-1979ae1c15f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:47.719445 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-8a52a5a6-691c-4bb3-8134-0d1b235ecd74 None None] MFA Rules not processed for user `b6880278f7304051b9ef03e927637e74`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:47.724745 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-8a52a5a6-691c-4bb3-8134-0d1b235ecd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:47.727220 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 915/1810] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:47.754606 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda8121-d8e5-4f7d-8d1b-1979ae1c15f0 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:47.756139 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-abda8121-d8e5-4f7d-8d1b-1979ae1c15f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:47.761595 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:47.830147 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 896/1811] 10.222.0.98 () {58 vars in 1009 bytes} [Thu Aug 18 12:20:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:47.878245 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda8121-d8e5-4f7d-8d1b-1979ae1c15f0 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:47.882978 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4709f043-94bf-409e-88df-12bef96c615c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:47.883934 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709f043-94bf-409e-88df-12bef96c615c service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:47.884432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709f043-94bf-409e-88df-12bef96c615c service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:47.884938 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709f043-94bf-409e-88df-12bef96c615c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:47.900672 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4709f043-94bf-409e-88df-12bef96c615c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:47.902823 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4709f043-94bf-409e-88df-12bef96c615c service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:47.932203 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 916/1812] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:20:48.256756 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a52a5a6-691c-4bb3-8134-0d1b235ecd74 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:48.258699 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-8a52a5a6-691c-4bb3-8134-0d1b235ecd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:48.383586 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a52a5a6-691c-4bb3-8134-0d1b235ecd74 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:48.394662 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d424a-9fec-4893-a6f2-49d9e0feda57 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771591fe31cb4e8c9c4320c2a7b2719d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:48.410856 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d424a-9fec-4893-a6f2-49d9e0feda57 service swift] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:48.413735 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d424a-9fec-4893-a6f2-49d9e0feda57 service swift] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:48.421782 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d424a-9fec-4893-a6f2-49d9e0feda57 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:48.446346 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03d424a-9fec-4893-a6f2-49d9e0feda57 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:48.449999 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03d424a-9fec-4893-a6f2-49d9e0feda57 service swift] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:48.458876 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:48.470927 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 897/1813] 10.222.0.98 () {60 vars in 1384 bytes} [Thu Aug 18 12:20:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 12:20:49.374640 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df1139b-2b56-4d81-8edb-f49edfd2e275 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:49.375259 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df1139b-2b56-4d81-8edb-f49edfd2e275 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:49.375729 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df1139b-2b56-4d81-8edb-f49edfd2e275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:49.434563 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-4df1139b-2b56-4d81-8edb-f49edfd2e275 None None] MFA Rules not processed for user `185644affb284c2b91e604cf99803ffb`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:49.436556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-4df1139b-2b56-4d81-8edb-f49edfd2e275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:49.464836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 917/1814] 10.222.0.76 () {58 vars in 1015 bytes} [Thu Aug 18 12:20:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:20:49.523251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d424a-9fec-4893-a6f2-49d9e0feda57 service swift] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:49.525432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df1139b-2b56-4d81-8edb-f49edfd2e275 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:49.527096 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-4df1139b-2b56-4d81-8edb-f49edfd2e275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:49.542360 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d424a-9fec-4893-a6f2-49d9e0feda57 service swift] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:49.547937 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b067534-ee69-4dd2-9f84-b3d8b1c72624 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:49.549530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b067534-ee69-4dd2-9f84-b3d8b1c72624 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:49.550023 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b067534-ee69-4dd2-9f84-b3d8b1c72624 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:49.550639 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b067534-ee69-4dd2-9f84-b3d8b1c72624 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:49.563280 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b067534-ee69-4dd2-9f84-b3d8b1c72624 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:49.566326 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df1139b-2b56-4d81-8edb-f49edfd2e275 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:49.578405 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c5c2e-0953-4b7a-a9c9-36978ee7bc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:49.579327 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c5c2e-0953-4b7a-a9c9-36978ee7bc3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:49.579910 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c5c2e-0953-4b7a-a9c9-36978ee7bc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:49.580708 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c5c2e-0953-4b7a-a9c9-36978ee7bc3d admin admin] PATH_INFO: `/v3/projects/21b90878aee347a08b8cc639839f4f46` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:49.581373 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b067534-ee69-4dd2-9f84-b3d8b1c72624 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:49.583633 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0c5c2e-0953-4b7a-a9c9-36978ee7bc3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=21b90878aee347a08b8cc639839f4f46)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:49.591124 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0c5c2e-0953-4b7a-a9c9-36978ee7bc3d admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:49.645876 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 898/1815] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:20:49.939138 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-3b0c5c2e-0953-4b7a-a9c9-36978ee7bc3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21b90878aee347a08b8cc639839f4f46', 'request_id': 'req-3b0c5c2e-0953-4b7a-a9c9-36978ee7bc3d'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:49.962346 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-3b0c5c2e-0953-4b7a-a9c9-36978ee7bc3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21b90878aee347a08b8cc639839f4f46', 'request_id': 'req-3b0c5c2e-0953-4b7a-a9c9-36978ee7bc3d'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:50.226628 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-3b0c5c2e-0953-4b7a-a9c9-36978ee7bc3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21b90878aee347a08b8cc639839f4f46', 'request_id': 'req-3b0c5c2e-0953-4b7a-a9c9-36978ee7bc3d'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:50.255761 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-3b0c5c2e-0953-4b7a-a9c9-36978ee7bc3d admin admin] The token cache is being invalidate because project 21b90878aee347a08b8cc639839f4f46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:20:50.256388 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-3b0c5c2e-0953-4b7a-a9c9-36978ee7bc3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:50.257920 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-3b0c5c2e-0953-4b7a-a9c9-36978ee7bc3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:50.264415 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 918/1816] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:20:49 2022] DELETE /identity/v3/projects/21b90878aee347a08b8cc639839f4f46 => generated 0 bytes in 739 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:50.268439 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b067534-ee69-4dd2-9f84-b3d8b1c72624 service neutron] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:50.285751 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b067534-ee69-4dd2-9f84-b3d8b1c72624 service neutron] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:50.493826 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3b320-c139-4b1c-8583-a822414da3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:50.495573 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3b320-c139-4b1c-8583-a822414da3bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:50.497894 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3b320-c139-4b1c-8583-a822414da3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:50.498354 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3b320-c139-4b1c-8583-a822414da3bb admin admin] PATH_INFO: `/v3/users/bf4b6c4ff99c429d8a0d8459fe1877a0` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:50.508073 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c3b320-c139-4b1c-8583-a822414da3bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf4b6c4ff99c429d8a0d8459fe1877a0)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:50.508073 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c3b320-c139-4b1c-8583-a822414da3bb admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:50.578487 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c5c2e-0953-4b7a-a9c9-36978ee7bc3d admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:50.579966 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-3b0c5c2e-0953-4b7a-a9c9-36978ee7bc3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:50.614803 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c5c2e-0953-4b7a-a9c9-36978ee7bc3d admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:50.642144 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c91ba-bdb1-433c-b440-f0e4e891020e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:50.650911 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c91ba-bdb1-433c-b440-f0e4e891020e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:50.651252 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c91ba-bdb1-433c-b440-f0e4e891020e admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:50.651566 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c91ba-bdb1-433c-b440-f0e4e891020e admin admin] PATH_INFO: `/v3/users/12c8289effa743009a910410a3d0facd` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:50.655266 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c91ba-bdb1-433c-b440-f0e4e891020e admin admin] RBAC: Authorizing `identity:delete_user(user_id=12c8289effa743009a910410a3d0facd)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:50.657720 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c91ba-bdb1-433c-b440-f0e4e891020e admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:50.940112 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-a2c3b320-c139-4b1c-8583-a822414da3bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf4b6c4ff99c429d8a0d8459fe1877a0'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:50.943968 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-a2c3b320-c139-4b1c-8583-a822414da3bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf4b6c4ff99c429d8a0d8459fe1877a0'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:50.946898 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-a2c3b320-c139-4b1c-8583-a822414da3bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf4b6c4ff99c429d8a0d8459fe1877a0'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:50.948939 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-8a8c91ba-bdb1-433c-b440-f0e4e891020e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12c8289effa743009a910410a3d0facd'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:50.962149 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-8a8c91ba-bdb1-433c-b440-f0e4e891020e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12c8289effa743009a910410a3d0facd'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:50.963534 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-8a8c91ba-bdb1-433c-b440-f0e4e891020e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12c8289effa743009a910410a3d0facd'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:50.967443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-a2c3b320-c139-4b1c-8583-a822414da3bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf4b6c4ff99c429d8a0d8459fe1877a0'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:51.001452 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-a2c3b320-c139-4b1c-8583-a822414da3bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf4b6c4ff99c429d8a0d8459fe1877a0'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:51.007794 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-8a8c91ba-bdb1-433c-b440-f0e4e891020e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12c8289effa743009a910410a3d0facd'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:51.011667 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:51.106902 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 899/1817] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:20:50 2022] DELETE /identity/v3/users/bf4b6c4ff99c429d8a0d8459fe1877a0 => generated 0 bytes in 839 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:51.157284 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-8a8c91ba-bdb1-433c-b440-f0e4e891020e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12c8289effa743009a910410a3d0facd'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:51.169384 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 919/1818] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:20:50 2022] DELETE /identity/v3/users/12c8289effa743009a910410a3d0facd => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:51.174170 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e264e-bd3a-48c3-91bb-93bd74ca8345 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:51.174470 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e264e-bd3a-48c3-91bb-93bd74ca8345 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:51.174746 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e264e-bd3a-48c3-91bb-93bd74ca8345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:51.223683 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-eb6e264e-bd3a-48c3-91bb-93bd74ca8345 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:51.227378 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-eb6e264e-bd3a-48c3-91bb-93bd74ca8345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:51.407384 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 900/1819] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:51.440941 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c91ba-bdb1-433c-b440-f0e4e891020e admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:51.442785 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-8a8c91ba-bdb1-433c-b440-f0e4e891020e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:51.585036 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c91ba-bdb1-433c-b440-f0e4e891020e admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:51.593045 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd3273-821c-47f3-a04f-ed57f2902ac7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:51.594532 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd3273-821c-47f3-a04f-ed57f2902ac7 service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:51.594990 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd3273-821c-47f3-a04f-ed57f2902ac7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:51.595642 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd3273-821c-47f3-a04f-ed57f2902ac7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:51.632175 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd3273-821c-47f3-a04f-ed57f2902ac7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:51.634401 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd3273-821c-47f3-a04f-ed57f2902ac7 service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:51.665655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 920/1820] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:20:52.219791 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40dcad3-4206-493e-9c25-e0cdbe08a966 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:52.225819 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40dcad3-4206-493e-9c25-e0cdbe08a966 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:52.225819 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40dcad3-4206-493e-9c25-e0cdbe08a966 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:52.225819 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 901/1821] 10.222.0.98 () {54 vars in 921 bytes} [Thu Aug 18 12:20:52 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:20:52.512740 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e4c959-7cda-4481-8ccf-5d5bb449df0e None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:52.513027 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e4c959-7cda-4481-8ccf-5d5bb449df0e None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:52.515004 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e4c959-7cda-4481-8ccf-5d5bb449df0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:52.602095 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-52e4c959-7cda-4481-8ccf-5d5bb449df0e None None] MFA Rules not processed for user `171c72a6b29e418bbb4ea5c6faffe3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:52.638214 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-52e4c959-7cda-4481-8ccf-5d5bb449df0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:52.794384 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40dcad3-4206-493e-9c25-e0cdbe08a966 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:52.803756 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-c40dcad3-4206-493e-9c25-e0cdbe08a966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:52.837337 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40dcad3-4206-493e-9c25-e0cdbe08a966 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:52.839207 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 921/1822] 10.222.0.98 () {58 vars in 1018 bytes} [Thu Aug 18 12:20:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:52.858385 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987707b5-0783-4af2-9d29-8d3b580eda12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:52.859111 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987707b5-0783-4af2-9d29-8d3b580eda12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:52.859381 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987707b5-0783-4af2-9d29-8d3b580eda12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:52.859879 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987707b5-0783-4af2-9d29-8d3b580eda12 admin admin] PATH_INFO: `/v3/users/432509e0a09b40a0b4366334e7c229d5` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:52.861183 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987707b5-0783-4af2-9d29-8d3b580eda12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=432509e0a09b40a0b4366334e7c229d5)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:52.863182 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987707b5-0783-4af2-9d29-8d3b580eda12 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:52.865204 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:52.938425 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e4c959-7cda-4481-8ccf-5d5bb449df0e None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:52.939993 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-52e4c959-7cda-4481-8ccf-5d5bb449df0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:52.960196 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e4c959-7cda-4481-8ccf-5d5bb449df0e None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:52.984428 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842adcf6-c40d-4353-9d63-3fc2f1599a6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:52.985472 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842adcf6-c40d-4353-9d63-3fc2f1599a6b service nova] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:52.986182 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842adcf6-c40d-4353-9d63-3fc2f1599a6b service nova] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:52.986718 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842adcf6-c40d-4353-9d63-3fc2f1599a6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:53.001982 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842adcf6-c40d-4353-9d63-3fc2f1599a6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:53.005003 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842adcf6-c40d-4353-9d63-3fc2f1599a6b service nova] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:53.019452 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 922/1823] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:20:53.028003 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13181a-d247-49a7-9c68-8ce960d81c29 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:53.029295 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13181a-d247-49a7-9c68-8ce960d81c29 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:53.029783 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13181a-d247-49a7-9c68-8ce960d81c29 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:53.031525 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 923/1824] 10.222.0.98 () {54 vars in 921 bytes} [Thu Aug 18 12:20:53 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:20:53.217883 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-987707b5-0783-4af2-9d29-8d3b580eda12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '432509e0a09b40a0b4366334e7c229d5'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:53.221650 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-987707b5-0783-4af2-9d29-8d3b580eda12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '432509e0a09b40a0b4366334e7c229d5'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:53.225713 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-987707b5-0783-4af2-9d29-8d3b580eda12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '432509e0a09b40a0b4366334e7c229d5'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:53.252479 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-987707b5-0783-4af2-9d29-8d3b580eda12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '432509e0a09b40a0b4366334e7c229d5'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:53.287866 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-987707b5-0783-4af2-9d29-8d3b580eda12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '432509e0a09b40a0b4366334e7c229d5'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:53.336226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a591b0-cbca-4573-bca7-e1858f06468f None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:53.336606 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a591b0-cbca-4573-bca7-e1858f06468f None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:53.336606 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a591b0-cbca-4573-bca7-e1858f06468f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:53.388441 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-23a591b0-cbca-4573-bca7-e1858f06468f None None] MFA Rules not processed for user `171c72a6b29e418bbb4ea5c6faffe3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:53.394851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-23a591b0-cbca-4573-bca7-e1858f06468f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:53.400657 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 902/1825] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:20:52 2022] DELETE /identity/v3/users/432509e0a09b40a0b4366334e7c229d5 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:53.409438 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff37f6-5449-446e-b1dc-e17840a37f93 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:53.409810 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff37f6-5449-446e-b1dc-e17840a37f93 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:53.410154 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff37f6-5449-446e-b1dc-e17840a37f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:53.440981 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-ecff37f6-5449-446e-b1dc-e17840a37f93 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:53.442747 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-ecff37f6-5449-446e-b1dc-e17840a37f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:53.587281 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 924/1826] 10.222.0.98 () {58 vars in 1018 bytes} [Thu Aug 18 12:20:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:20:53.599802 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 903/1827] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:53.634482 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a591b0-cbca-4573-bca7-e1858f06468f None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:53.636798 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-23a591b0-cbca-4573-bca7-e1858f06468f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:53.693316 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecff37f6-5449-446e-b1dc-e17840a37f93 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:53.696949 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-ecff37f6-5449-446e-b1dc-e17840a37f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:53.729015 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecff37f6-5449-446e-b1dc-e17840a37f93 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:53.756143 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dec281-da19-4fa4-9206-1818bccb2215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:53.757181 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dec281-da19-4fa4-9206-1818bccb2215 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:53.759647 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dec281-da19-4fa4-9206-1818bccb2215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:53.760325 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dec281-da19-4fa4-9206-1818bccb2215 admin admin] PATH_INFO: `/v3/projects/f4e8b88a3d7c44048186ed3302738ffc` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:53.766339 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dec281-da19-4fa4-9206-1818bccb2215 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4e8b88a3d7c44048186ed3302738ffc)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:53.770598 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dec281-da19-4fa4-9206-1818bccb2215 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:53.896635 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a591b0-cbca-4573-bca7-e1858f06468f None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:53.904224 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6597a5-9c62-4e35-8e24-5b30367d5d30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:53.905268 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6597a5-9c62-4e35-8e24-5b30367d5d30 service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:53.905507 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6597a5-9c62-4e35-8e24-5b30367d5d30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:53.905841 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6597a5-9c62-4e35-8e24-5b30367d5d30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:53.920343 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6597a5-9c62-4e35-8e24-5b30367d5d30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:53.922170 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6597a5-9c62-4e35-8e24-5b30367d5d30 service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:53.949323 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 925/1828] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:20:53.960694 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6597a5-9c62-4e35-8e24-5b30367d5d30 service neutron] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:53.964453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-5f6597a5-9c62-4e35-8e24-5b30367d5d30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:54.020126 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6597a5-9c62-4e35-8e24-5b30367d5d30 service neutron] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:54.105360 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-50dec281-da19-4fa4-9206-1818bccb2215 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4e8b88a3d7c44048186ed3302738ffc', 'request_id': 'req-50dec281-da19-4fa4-9206-1818bccb2215'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:54.130606 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-50dec281-da19-4fa4-9206-1818bccb2215 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4e8b88a3d7c44048186ed3302738ffc', 'request_id': 'req-50dec281-da19-4fa4-9206-1818bccb2215'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:54.209501 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e974eea0-944f-4f24-ba7b-d2325413ac7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:54.211484 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e974eea0-944f-4f24-ba7b-d2325413ac7b service nova] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:54.217580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e974eea0-944f-4f24-ba7b-d2325413ac7b service nova] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:54.217580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e974eea0-944f-4f24-ba7b-d2325413ac7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:54.234497 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e974eea0-944f-4f24-ba7b-d2325413ac7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:54.237206 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e974eea0-944f-4f24-ba7b-d2325413ac7b service nova] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:54.240413 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-50dec281-da19-4fa4-9206-1818bccb2215 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4e8b88a3d7c44048186ed3302738ffc', 'request_id': 'req-50dec281-da19-4fa4-9206-1818bccb2215'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:54.257456 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 926/1829] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:20:54.263087 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-50dec281-da19-4fa4-9206-1818bccb2215 admin admin] The token cache is being invalidate because project f4e8b88a3d7c44048186ed3302738ffc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:20:54.265055 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e974eea0-944f-4f24-ba7b-d2325413ac7b service nova] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:54.268953 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-50dec281-da19-4fa4-9206-1818bccb2215 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:54.269082 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-e974eea0-944f-4f24-ba7b-d2325413ac7b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:54.269504 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-50dec281-da19-4fa4-9206-1818bccb2215 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:54.276047 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 904/1830] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:20:53 2022] DELETE /identity/v3/projects/f4e8b88a3d7c44048186ed3302738ffc => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:54.281704 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89e0df-6bc3-4635-8623-fd0e3ce8a5a6 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:54.282104 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89e0df-6bc3-4635-8623-fd0e3ce8a5a6 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:54.282494 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89e0df-6bc3-4635-8623-fd0e3ce8a5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:54.374072 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-0b89e0df-6bc3-4635-8623-fd0e3ce8a5a6 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:54.375438 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-0b89e0df-6bc3-4635-8623-fd0e3ce8a5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:54.376238 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e974eea0-944f-4f24-ba7b-d2325413ac7b service nova] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:54.384696 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c22b68-035d-4ebb-ae6d-cd3cdfc73463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:54.386355 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c22b68-035d-4ebb-ae6d-cd3cdfc73463 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:54.386714 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c22b68-035d-4ebb-ae6d-cd3cdfc73463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:54.387061 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c22b68-035d-4ebb-ae6d-cd3cdfc73463 admin admin] PATH_INFO: `/v3/projects/5c57cdf8e2c14790b249a67d360deeb4` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:54.390027 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c22b68-035d-4ebb-ae6d-cd3cdfc73463 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c57cdf8e2c14790b249a67d360deeb4)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:54.391845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c22b68-035d-4ebb-ae6d-cd3cdfc73463 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:54.398491 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 905/1831] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:20:54.408213 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2aef67-0856-4b3f-9435-ecccd409526d None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:54.408725 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2aef67-0856-4b3f-9435-ecccd409526d None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:54.409250 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2aef67-0856-4b3f-9435-ecccd409526d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:54.452846 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-5a2aef67-0856-4b3f-9435-ecccd409526d None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:54.457215 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-5a2aef67-0856-4b3f-9435-ecccd409526d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:54.500539 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 906/1832] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:20:54.503464 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2aef67-0856-4b3f-9435-ecccd409526d None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:54.512121 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:54.530085 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2aef67-0856-4b3f-9435-ecccd409526d None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:54.537839 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeccef11-ef2c-42fe-9386-2fcc3fcdb479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:54.539002 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeccef11-ef2c-42fe-9386-2fcc3fcdb479 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:54.539620 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeccef11-ef2c-42fe-9386-2fcc3fcdb479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:54.540145 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeccef11-ef2c-42fe-9386-2fcc3fcdb479 admin admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:54.541047 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeccef11-ef2c-42fe-9386-2fcc3fcdb479 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:20:54.559206 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeccef11-ef2c-42fe-9386-2fcc3fcdb479 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:54.562070 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeccef11-ef2c-42fe-9386-2fcc3fcdb479 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:54.578373 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 907/1833] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:20:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:20:54.582783 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeccef11-ef2c-42fe-9386-2fcc3fcdb479 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:54.601833 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeccef11-ef2c-42fe-9386-2fcc3fcdb479 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:54.609291 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5473864a-182b-4af9-a700-7952cf94a903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:54.610114 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473864a-182b-4af9-a700-7952cf94a903 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:54.610495 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473864a-182b-4af9-a700-7952cf94a903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:54.611314 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473864a-182b-4af9-a700-7952cf94a903 admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:54.612408 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5473864a-182b-4af9-a700-7952cf94a903 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:54.618134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5473864a-182b-4af9-a700-7952cf94a903 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:54.704698 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-89c22b68-035d-4ebb-ae6d-cd3cdfc73463 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c57cdf8e2c14790b249a67d360deeb4', 'request_id': 'req-89c22b68-035d-4ebb-ae6d-cd3cdfc73463'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:54.713771 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-89c22b68-035d-4ebb-ae6d-cd3cdfc73463 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c57cdf8e2c14790b249a67d360deeb4', 'request_id': 'req-89c22b68-035d-4ebb-ae6d-cd3cdfc73463'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:54.802551 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 908/1834] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:20:54 2022] POST /identity/v3/projects => generated 371 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:54.806745 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5473864a-182b-4af9-a700-7952cf94a903 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:54.846097 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-89c22b68-035d-4ebb-ae6d-cd3cdfc73463 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c57cdf8e2c14790b249a67d360deeb4', 'request_id': 'req-89c22b68-035d-4ebb-ae6d-cd3cdfc73463'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:54.878095 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-89c22b68-035d-4ebb-ae6d-cd3cdfc73463 admin admin] The token cache is being invalidate because project 5c57cdf8e2c14790b249a67d360deeb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:20:54.878396 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-89c22b68-035d-4ebb-ae6d-cd3cdfc73463 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:54.880514 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-89c22b68-035d-4ebb-ae6d-cd3cdfc73463 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:54.886264 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 927/1835] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:20:54 2022] DELETE /identity/v3/projects/5c57cdf8e2c14790b249a67d360deeb4 => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:54.888022 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c22b68-035d-4ebb-ae6d-cd3cdfc73463 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:54.892333 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-89c22b68-035d-4ebb-ae6d-cd3cdfc73463 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:55.006429 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5473864a-182b-4af9-a700-7952cf94a903 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:55.013532 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59c578-9680-4101-8986-9e5821e7e807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:55.016215 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59c578-9680-4101-8986-9e5821e7e807 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:55.017901 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59c578-9680-4101-8986-9e5821e7e807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:55.018039 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59c578-9680-4101-8986-9e5821e7e807 admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:55.019075 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a59c578-9680-4101-8986-9e5821e7e807 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:55.034084 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a59c578-9680-4101-8986-9e5821e7e807 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:55.057703 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c22b68-035d-4ebb-ae6d-cd3cdfc73463 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:55.068557 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d99c18d-5119-41f8-afde-03ac3c879ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:55.069317 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99c18d-5119-41f8-afde-03ac3c879ad6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:55.079834 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99c18d-5119-41f8-afde-03ac3c879ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:55.081540 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99c18d-5119-41f8-afde-03ac3c879ad6 admin admin] PATH_INFO: `/v3/users/b7f41e5aac4f466a93003b9238746f9b` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:55.083191 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d99c18d-5119-41f8-afde-03ac3c879ad6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7f41e5aac4f466a93003b9238746f9b)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:55.085605 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d99c18d-5119-41f8-afde-03ac3c879ad6 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:55.140577 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 909/1836] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:20:54 2022] POST /identity/v3/users => generated 286 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:55.144494 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59c578-9680-4101-8986-9e5821e7e807 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:55.149584 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-9a59c578-9680-4101-8986-9e5821e7e807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:55.171718 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59c578-9680-4101-8986-9e5821e7e807 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:55.176317 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191e88e-3480-4bf1-88eb-a62f9aec57ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:55.177553 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191e88e-3480-4bf1-88eb-a62f9aec57ec admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:55.178134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191e88e-3480-4bf1-88eb-a62f9aec57ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:55.178688 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191e88e-3480-4bf1-88eb-a62f9aec57ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:55.179857 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6191e88e-3480-4bf1-88eb-a62f9aec57ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:55.187679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6191e88e-3480-4bf1-88eb-a62f9aec57ec admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:55.218394 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 910/1837] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:55 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 12:20:55.220998 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191e88e-3480-4bf1-88eb-a62f9aec57ec admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:55.248958 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191e88e-3480-4bf1-88eb-a62f9aec57ec admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:55.271329 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665d3522-4ea8-48c9-b46a-1c2e12b167f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:55.272278 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d3522-4ea8-48c9-b46a-1c2e12b167f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:55.272715 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d3522-4ea8-48c9-b46a-1c2e12b167f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:55.273227 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d3522-4ea8-48c9-b46a-1c2e12b167f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:55.274085 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665d3522-4ea8-48c9-b46a-1c2e12b167f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:55.278493 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665d3522-4ea8-48c9-b46a-1c2e12b167f9 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:55.334432 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 911/1838] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:55 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:55.336154 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665d3522-4ea8-48c9-b46a-1c2e12b167f9 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:55.349013 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-5d99c18d-5119-41f8-afde-03ac3c879ad6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7f41e5aac4f466a93003b9238746f9b'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:55.378826 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665d3522-4ea8-48c9-b46a-1c2e12b167f9 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:55.384234 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-5d99c18d-5119-41f8-afde-03ac3c879ad6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7f41e5aac4f466a93003b9238746f9b'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:55.385514 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-5d99c18d-5119-41f8-afde-03ac3c879ad6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7f41e5aac4f466a93003b9238746f9b'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:55.402875 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b6059-5eb9-484e-8e65-d954b632442b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:55.403667 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b6059-5eb9-484e-8e65-d954b632442b admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:55.403958 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b6059-5eb9-484e-8e65-d954b632442b admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:55.404768 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b6059-5eb9-484e-8e65-d954b632442b admin admin] PATH_INFO: `/v3/projects/87abee0f1d844fa1a5e10a1b6690ab0c/users/32121bd691604ef0ac646d7491aee60a/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:55.420617 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:55.432090 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-5d99c18d-5119-41f8-afde-03ac3c879ad6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7f41e5aac4f466a93003b9238746f9b'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:55.461177 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3b6059-5eb9-484e-8e65-d954b632442b admin admin] RBAC: Authorizing `identity:create_grant(project_id=87abee0f1d844fa1a5e10a1b6690ab0c, user_id=32121bd691604ef0ac646d7491aee60a, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:55.464370 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3b6059-5eb9-484e-8e65-d954b632442b admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:55.512900 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:55.575924 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-5d99c18d-5119-41f8-afde-03ac3c879ad6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7f41e5aac4f466a93003b9238746f9b'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:55.590594 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 912/1839] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:20:55 2022] PUT /identity/v3/projects/87abee0f1d844fa1a5e10a1b6690ab0c/users/32121bd691604ef0ac646d7491aee60a/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:55.594373 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 928/1840] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:20:54 2022] DELETE /identity/v3/users/b7f41e5aac4f466a93003b9238746f9b => generated 0 bytes in 704 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:55.600988 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8fd869-ef1a-42b2-9328-f4f7c9bfc52f None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:55.601411 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8fd869-ef1a-42b2-9328-f4f7c9bfc52f None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:55.601780 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8fd869-ef1a-42b2-9328-f4f7c9bfc52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:55.680083 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-aa8fd869-ef1a-42b2-9328-f4f7c9bfc52f None None] MFA Rules not processed for user `32121bd691604ef0ac646d7491aee60a`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:55.683077 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-aa8fd869-ef1a-42b2-9328-f4f7c9bfc52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:55.874976 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 913/1841] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:20:55.928333 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d99c18d-5119-41f8-afde-03ac3c879ad6 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:55.928333 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-5d99c18d-5119-41f8-afde-03ac3c879ad6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:56.120837 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8fd869-ef1a-42b2-9328-f4f7c9bfc52f None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:56.123792 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-aa8fd869-ef1a-42b2-9328-f4f7c9bfc52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:56.173861 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d99c18d-5119-41f8-afde-03ac3c879ad6 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:56.206557 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d5ee72-7257-4ee4-a96b-48e5391de436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:56.207225 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d5ee72-7257-4ee4-a96b-48e5391de436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:56.207537 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d5ee72-7257-4ee4-a96b-48e5391de436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:56.207824 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d5ee72-7257-4ee4-a96b-48e5391de436 admin admin] PATH_INFO: `/v3/projects/d539003295e34a98bef641ed4186686c` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:56.213839 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d5ee72-7257-4ee4-a96b-48e5391de436 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d539003295e34a98bef641ed4186686c)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:56.222650 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d5ee72-7257-4ee4-a96b-48e5391de436 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:56.223697 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8fd869-ef1a-42b2-9328-f4f7c9bfc52f None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:56.227844 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5652623-f52d-467c-8f3d-62b070325a4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:56.228462 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5652623-f52d-467c-8f3d-62b070325a4a service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:56.228797 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5652623-f52d-467c-8f3d-62b070325a4a service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:56.229136 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5652623-f52d-467c-8f3d-62b070325a4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:56.239663 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5652623-f52d-467c-8f3d-62b070325a4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:56.241370 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5652623-f52d-467c-8f3d-62b070325a4a service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:56.254505 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 914/1842] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:20:56 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:20:56.420259 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-01d5ee72-7257-4ee4-a96b-48e5391de436 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd539003295e34a98bef641ed4186686c', 'request_id': 'req-01d5ee72-7257-4ee4-a96b-48e5391de436'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:56.433667 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-01d5ee72-7257-4ee4-a96b-48e5391de436 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd539003295e34a98bef641ed4186686c', 'request_id': 'req-01d5ee72-7257-4ee4-a96b-48e5391de436'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:56.522710 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-01d5ee72-7257-4ee4-a96b-48e5391de436 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd539003295e34a98bef641ed4186686c', 'request_id': 'req-01d5ee72-7257-4ee4-a96b-48e5391de436'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:56.558447 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-01d5ee72-7257-4ee4-a96b-48e5391de436 admin admin] The token cache is being invalidate because project d539003295e34a98bef641ed4186686c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:20:56.558746 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-01d5ee72-7257-4ee4-a96b-48e5391de436 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:56.563555 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-01d5ee72-7257-4ee4-a96b-48e5391de436 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:56.571001 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 929/1843] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:20:55 2022] DELETE /identity/v3/projects/d539003295e34a98bef641ed4186686c => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:56.574646 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5652623-f52d-467c-8f3d-62b070325a4a service cinder] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:56.609604 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5652623-f52d-467c-8f3d-62b070325a4a service cinder] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:56.759393 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba0e1c6-86a8-4095-933f-be4aa5de0593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:56.763051 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0e1c6-86a8-4095-933f-be4aa5de0593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:56.763904 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0e1c6-86a8-4095-933f-be4aa5de0593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:56.764363 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0e1c6-86a8-4095-933f-be4aa5de0593 admin admin] PATH_INFO: `/v3/users/9aea0547b9334314bd61be0f6cf90299` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:56.766224 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba0e1c6-86a8-4095-933f-be4aa5de0593 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9aea0547b9334314bd61be0f6cf90299)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:56.769617 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba0e1c6-86a8-4095-933f-be4aa5de0593 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:57.096617 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-9ba0e1c6-86a8-4095-933f-be4aa5de0593 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9aea0547b9334314bd61be0f6cf90299'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:57.097641 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-9ba0e1c6-86a8-4095-933f-be4aa5de0593 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9aea0547b9334314bd61be0f6cf90299'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:57.099831 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-9ba0e1c6-86a8-4095-933f-be4aa5de0593 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9aea0547b9334314bd61be0f6cf90299'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:57.125366 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-9ba0e1c6-86a8-4095-933f-be4aa5de0593 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9aea0547b9334314bd61be0f6cf90299'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:57.162509 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-9ba0e1c6-86a8-4095-933f-be4aa5de0593 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9aea0547b9334314bd61be0f6cf90299'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:57.331476 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 915/1844] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:20:56 2022] DELETE /identity/v3/users/9aea0547b9334314bd61be0f6cf90299 => generated 0 bytes in 757 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:57.548957 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:57.714277 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d5ee72-7257-4ee4-a96b-48e5391de436 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:57.715986 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-01d5ee72-7257-4ee4-a96b-48e5391de436 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:57.738771 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d5ee72-7257-4ee4-a96b-48e5391de436 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:57.885781 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f7c82b-e253-4ef1-ab30-7b77e388b6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:57.885781 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7c82b-e253-4ef1-ab30-7b77e388b6bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:57.886182 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7c82b-e253-4ef1-ab30-7b77e388b6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:57.886615 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7c82b-e253-4ef1-ab30-7b77e388b6bb admin admin] PATH_INFO: `/v3/projects/e0b3186e766d48dfa5272547c48d9a55` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:57.891127 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f7c82b-e253-4ef1-ab30-7b77e388b6bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0b3186e766d48dfa5272547c48d9a55)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:57.896279 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f7c82b-e253-4ef1-ab30-7b77e388b6bb admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:58.077449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-88f7c82b-e253-4ef1-ab30-7b77e388b6bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0b3186e766d48dfa5272547c48d9a55', 'request_id': 'req-88f7c82b-e253-4ef1-ab30-7b77e388b6bb'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:58.105405 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-88f7c82b-e253-4ef1-ab30-7b77e388b6bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0b3186e766d48dfa5272547c48d9a55', 'request_id': 'req-88f7c82b-e253-4ef1-ab30-7b77e388b6bb'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:58.267906 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-88f7c82b-e253-4ef1-ab30-7b77e388b6bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0b3186e766d48dfa5272547c48d9a55', 'request_id': 'req-88f7c82b-e253-4ef1-ab30-7b77e388b6bb'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:58.296810 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-88f7c82b-e253-4ef1-ab30-7b77e388b6bb admin admin] The token cache is being invalidate because project e0b3186e766d48dfa5272547c48d9a55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:20:58.297182 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-88f7c82b-e253-4ef1-ab30-7b77e388b6bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:58.298787 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-88f7c82b-e253-4ef1-ab30-7b77e388b6bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:58.301963 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 930/1845] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:20:57 2022] DELETE /identity/v3/projects/e0b3186e766d48dfa5272547c48d9a55 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:58.306884 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8685bf6-2822-4415-a38f-2d1560bc7c60 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:58.307397 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8685bf6-2822-4415-a38f-2d1560bc7c60 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:58.307859 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8685bf6-2822-4415-a38f-2d1560bc7c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:58.333893 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-f8685bf6-2822-4415-a38f-2d1560bc7c60 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:58.335980 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-f8685bf6-2822-4415-a38f-2d1560bc7c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:58.424900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 916/1846] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:58.431227 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3800c0-2bfc-4ae3-847a-b3a7d7e01cd1 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:58.431786 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3800c0-2bfc-4ae3-847a-b3a7d7e01cd1 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:58.432252 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3800c0-2bfc-4ae3-847a-b3a7d7e01cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:58.446072 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8685bf6-2822-4415-a38f-2d1560bc7c60 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:58.450045 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-f8685bf6-2822-4415-a38f-2d1560bc7c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:58.456211 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-cf3800c0-2bfc-4ae3-847a-b3a7d7e01cd1 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:58.457453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-cf3800c0-2bfc-4ae3-847a-b3a7d7e01cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:58.476591 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 931/1847] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:58.476727 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:20:58.481438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3800c0-2bfc-4ae3-847a-b3a7d7e01cd1 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:58.484258 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8685bf6-2822-4415-a38f-2d1560bc7c60 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:58.505165 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e40fdd6-4a81-4870-886e-267f9695945b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:58.505165 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40fdd6-4a81-4870-886e-267f9695945b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:58.505165 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40fdd6-4a81-4870-886e-267f9695945b admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:58.505823 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3800c0-2bfc-4ae3-847a-b3a7d7e01cd1 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:58.506477 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40fdd6-4a81-4870-886e-267f9695945b admin admin] PATH_INFO: `/v3/projects/1e4ad0eb37e4424d8fd4492518a88369` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:58.512724 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e40fdd6-4a81-4870-886e-267f9695945b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e4ad0eb37e4424d8fd4492518a88369)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:58.512993 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82d1aec-fe55-44b3-bfed-548e15ca52eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:58.514549 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d1aec-fe55-44b3-bfed-548e15ca52eb admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:58.515395 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d1aec-fe55-44b3-bfed-548e15ca52eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:58.515395 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d1aec-fe55-44b3-bfed-548e15ca52eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:58.515855 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82d1aec-fe55-44b3-bfed-548e15ca52eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:20:58.516356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82d1aec-fe55-44b3-bfed-548e15ca52eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:58.519164 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82d1aec-fe55-44b3-bfed-548e15ca52eb admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:58.519946 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e40fdd6-4a81-4870-886e-267f9695945b admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:58.557943 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 932/1848] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:20:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:20:58.564842 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82d1aec-fe55-44b3-bfed-548e15ca52eb admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:58.638251 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82d1aec-fe55-44b3-bfed-548e15ca52eb admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:58.656182 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1540ad-f3d5-4b42-a245-31e2955d8d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:58.656783 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1540ad-f3d5-4b42-a245-31e2955d8d61 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:58.657069 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1540ad-f3d5-4b42-a245-31e2955d8d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:58.660448 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1540ad-f3d5-4b42-a245-31e2955d8d61 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:58.662164 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1540ad-f3d5-4b42-a245-31e2955d8d61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:58.667892 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1540ad-f3d5-4b42-a245-31e2955d8d61 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:58.702777 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-0e40fdd6-4a81-4870-886e-267f9695945b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e4ad0eb37e4424d8fd4492518a88369', 'request_id': 'req-0e40fdd6-4a81-4870-886e-267f9695945b'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:58.718770 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-0e40fdd6-4a81-4870-886e-267f9695945b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e4ad0eb37e4424d8fd4492518a88369', 'request_id': 'req-0e40fdd6-4a81-4870-886e-267f9695945b'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:58.789292 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 933/1849] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:20:58 2022] POST /identity/v3/projects => generated 387 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:58.800965 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1540ad-f3d5-4b42-a245-31e2955d8d61 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:58.849973 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-0e40fdd6-4a81-4870-886e-267f9695945b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e4ad0eb37e4424d8fd4492518a88369', 'request_id': 'req-0e40fdd6-4a81-4870-886e-267f9695945b'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:58.871021 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-0e40fdd6-4a81-4870-886e-267f9695945b admin admin] The token cache is being invalidate because project 1e4ad0eb37e4424d8fd4492518a88369 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:20:58.871607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-0e40fdd6-4a81-4870-886e-267f9695945b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:58.879367 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-0e40fdd6-4a81-4870-886e-267f9695945b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:20:58.891408 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 917/1850] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:20:58 2022] DELETE /identity/v3/projects/1e4ad0eb37e4424d8fd4492518a88369 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:20:58.897982 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87f03f-dd26-474f-9d93-3f42cff5c439 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:58.898502 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87f03f-dd26-474f-9d93-3f42cff5c439 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:58.898887 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87f03f-dd26-474f-9d93-3f42cff5c439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:58.934116 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1540ad-f3d5-4b42-a245-31e2955d8d61 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:58.945322 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7daf50e-275c-4b1c-b1db-54f2ba7bfcb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:58.946176 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7daf50e-275c-4b1c-b1db-54f2ba7bfcb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:58.946558 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7daf50e-275c-4b1c-b1db-54f2ba7bfcb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:58.947198 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7daf50e-275c-4b1c-b1db-54f2ba7bfcb0 admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:58.947906 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7daf50e-275c-4b1c-b1db-54f2ba7bfcb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:58.948048 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-1d87f03f-dd26-474f-9d93-3f42cff5c439 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:58.951338 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7daf50e-275c-4b1c-b1db-54f2ba7bfcb0 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:58.952679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-1d87f03f-dd26-474f-9d93-3f42cff5c439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:58.981417 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 918/1851] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:20:58.993044 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154c0d5-5e4b-46b3-8c4f-11f4bf15df04 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:58.993606 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154c0d5-5e4b-46b3-8c4f-11f4bf15df04 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:58.993983 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154c0d5-5e4b-46b3-8c4f-11f4bf15df04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:59.032093 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-b154c0d5-5e4b-46b3-8c4f-11f4bf15df04 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:59.034951 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-b154c0d5-5e4b-46b3-8c4f-11f4bf15df04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:59.052496 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 919/1852] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:20:59.058775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154c0d5-5e4b-46b3-8c4f-11f4bf15df04 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:59.075793 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 934/1853] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:20:58 2022] POST /identity/v3/users => generated 294 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:59.077197 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7daf50e-275c-4b1c-b1db-54f2ba7bfcb0 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:59.088089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-d7daf50e-275c-4b1c-b1db-54f2ba7bfcb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:59.112666 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154c0d5-5e4b-46b3-8c4f-11f4bf15df04 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:59.121867 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7daf50e-275c-4b1c-b1db-54f2ba7bfcb0 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:59.123508 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f9bffe-9d61-4033-9945-ba7614b1ca30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:59.124698 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9bffe-9d61-4033-9945-ba7614b1ca30 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:59.128804 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9bffe-9d61-4033-9945-ba7614b1ca30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:59.130123 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9bffe-9d61-4033-9945-ba7614b1ca30 admin admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:59.130749 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f9bffe-9d61-4033-9945-ba7614b1ca30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:20:59.131571 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f9bffe-9d61-4033-9945-ba7614b1ca30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:59.135918 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f9bffe-9d61-4033-9945-ba7614b1ca30 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:59.142961 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb584cb2-5f8e-45d4-a250-dba1ed3a5de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:59.144052 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb584cb2-5f8e-45d4-a250-dba1ed3a5de5 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:59.144358 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb584cb2-5f8e-45d4-a250-dba1ed3a5de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:59.144647 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb584cb2-5f8e-45d4-a250-dba1ed3a5de5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:59.145423 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb584cb2-5f8e-45d4-a250-dba1ed3a5de5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:59.147321 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb584cb2-5f8e-45d4-a250-dba1ed3a5de5 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:59.168754 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 920/1854] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:20:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:20:59.169179 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 935/1855] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:59.174100 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f9bffe-9d61-4033-9945-ba7614b1ca30 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:59.181482 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb584cb2-5f8e-45d4-a250-dba1ed3a5de5 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:59.198656 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f9bffe-9d61-4033-9945-ba7614b1ca30 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:59.206505 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb584cb2-5f8e-45d4-a250-dba1ed3a5de5 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:59.212363 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9f1e0-26b4-43e0-9c7d-5373249ee244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:59.213693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9f1e0-26b4-43e0-9c7d-5373249ee244 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:59.213843 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56818b7b-ffe5-4f40-9514-fd8b9f899bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:59.214306 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56818b7b-ffe5-4f40-9514-fd8b9f899bc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:59.214460 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56818b7b-ffe5-4f40-9514-fd8b9f899bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:59.215127 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56818b7b-ffe5-4f40-9514-fd8b9f899bc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:59.215697 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56818b7b-ffe5-4f40-9514-fd8b9f899bc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:59.217948 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56818b7b-ffe5-4f40-9514-fd8b9f899bc9 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:59.226986 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9f1e0-26b4-43e0-9c7d-5373249ee244 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:59.227481 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9f1e0-26b4-43e0-9c7d-5373249ee244 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:59.228721 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f9f1e0-26b4-43e0-9c7d-5373249ee244 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:59.230416 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f9f1e0-26b4-43e0-9c7d-5373249ee244 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:59.242622 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 921/1856] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:59.244821 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56818b7b-ffe5-4f40-9514-fd8b9f899bc9 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:59.268243 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56818b7b-ffe5-4f40-9514-fd8b9f899bc9 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:59.275941 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf6481-e599-4e1c-943c-7cb5b0c65fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:59.276693 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf6481-e599-4e1c-943c-7cb5b0c65fdb admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:59.277364 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf6481-e599-4e1c-943c-7cb5b0c65fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:59.277660 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf6481-e599-4e1c-943c-7cb5b0c65fdb admin admin] PATH_INFO: `/v3/projects/dbb4fe406c5f4d8ba0494e16688b263a/users/be38f43e768046e7a539991f9af44ed0/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:59.313937 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf6481-e599-4e1c-943c-7cb5b0c65fdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbb4fe406c5f4d8ba0494e16688b263a, user_id=be38f43e768046e7a539991f9af44ed0, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:59.318459 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf6481-e599-4e1c-943c-7cb5b0c65fdb admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:59.375562 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 936/1857] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:20:59 2022] POST /identity/v3/projects => generated 369 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:59.376881 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9f1e0-26b4-43e0-9c7d-5373249ee244 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:59.391808 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 922/1858] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:20:59 2022] PUT /identity/v3/projects/dbb4fe406c5f4d8ba0494e16688b263a/users/be38f43e768046e7a539991f9af44ed0/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:20:59.403804 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497528d-cbb2-45ad-8fe6-967163bcf8af None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:59.405029 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497528d-cbb2-45ad-8fe6-967163bcf8af None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:59.405540 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497528d-cbb2-45ad-8fe6-967163bcf8af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:59.466147 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-9497528d-cbb2-45ad-8fe6-967163bcf8af None None] MFA Rules not processed for user `be38f43e768046e7a539991f9af44ed0`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:20:59.468064 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-9497528d-cbb2-45ad-8fe6-967163bcf8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:20:59.476031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9f1e0-26b4-43e0-9c7d-5373249ee244 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:59.566003 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 923/1859] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:20:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:20:59.572056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9497528d-cbb2-45ad-8fe6-967163bcf8af None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:59.584103 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd272bb-31ca-4ec7-9ca2-177964b50bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:59.584986 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd272bb-31ca-4ec7-9ca2-177964b50bc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:59.585336 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd272bb-31ca-4ec7-9ca2-177964b50bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:59.585702 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd272bb-31ca-4ec7-9ca2-177964b50bc6 admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:59.586287 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd272bb-31ca-4ec7-9ca2-177964b50bc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:59.589080 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd272bb-31ca-4ec7-9ca2-177964b50bc6 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:59.670750 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9497528d-cbb2-45ad-8fe6-967163bcf8af None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:59.675565 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d212714-3b55-4e77-8176-9b27dfe6de99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:59.676202 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d212714-3b55-4e77-8176-9b27dfe6de99 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:59.676489 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d212714-3b55-4e77-8176-9b27dfe6de99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:59.676756 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d212714-3b55-4e77-8176-9b27dfe6de99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:59.677520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d212714-3b55-4e77-8176-9b27dfe6de99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:59.679662 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d212714-3b55-4e77-8176-9b27dfe6de99 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:59.766127 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 937/1860] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:20:59 2022] POST /identity/v3/users => generated 285 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:59.769906 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd272bb-31ca-4ec7-9ca2-177964b50bc6 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:59.795774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd272bb-31ca-4ec7-9ca2-177964b50bc6 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:59.802618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a034022-966d-4ab4-b733-e818da9c2fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:59.804096 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a034022-966d-4ab4-b733-e818da9c2fde admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:59.804975 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a034022-966d-4ab4-b733-e818da9c2fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:59.805873 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a034022-966d-4ab4-b733-e818da9c2fde admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:59.807837 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a034022-966d-4ab4-b733-e818da9c2fde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:59.810318 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a034022-966d-4ab4-b733-e818da9c2fde admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:59.813915 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 924/1861] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:20:59 2022] POST /identity/v3/projects => generated 387 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:20:59.817656 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d212714-3b55-4e77-8176-9b27dfe6de99 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:59.834526 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 938/1862] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:20:59.837482 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a034022-966d-4ab4-b733-e818da9c2fde admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:20:59.851746 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a034022-966d-4ab4-b733-e818da9c2fde admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:59.933064 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d212714-3b55-4e77-8176-9b27dfe6de99 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:20:59.957834 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9f2bc4-2125-430f-8e34-32b34c438b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:59.959356 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f2bc4-2125-430f-8e34-32b34c438b72 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:59.959356 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f2bc4-2125-430f-8e34-32b34c438b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:59.959356 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f2bc4-2125-430f-8e34-32b34c438b72 admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:59.965084 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9f2bc4-2125-430f-8e34-32b34c438b72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:59.970880 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9f2bc4-2125-430f-8e34-32b34c438b72 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:20:59.990135 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4a809-7457-4219-8736-75792c512f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:20:59.990883 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4a809-7457-4219-8736-75792c512f0f admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:20:59.991191 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4a809-7457-4219-8736-75792c512f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:20:59.991961 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4a809-7457-4219-8736-75792c512f0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:20:59.992464 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4a809-7457-4219-8736-75792c512f0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:20:59.994195 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4a809-7457-4219-8736-75792c512f0f admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:00.014477 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 939/1863] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:21:00.020454 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4a809-7457-4219-8736-75792c512f0f admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:00.051777 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4a809-7457-4219-8736-75792c512f0f admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:00.058151 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831eeacf-7a94-46f7-a4ad-06ed21dc6945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:00.059113 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831eeacf-7a94-46f7-a4ad-06ed21dc6945 admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:00.059513 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831eeacf-7a94-46f7-a4ad-06ed21dc6945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:00.059725 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831eeacf-7a94-46f7-a4ad-06ed21dc6945 admin admin] PATH_INFO: `/v3/projects/ed18129b40944cdda536f68868b860c1/users/f65a75a8587b42879cc0446fe1558225/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:00.066684 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 925/1864] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:20:59 2022] POST /identity/v3/users => generated 300 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:21:00.075377 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9f2bc4-2125-430f-8e34-32b34c438b72 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:00.101362 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9f2bc4-2125-430f-8e34-32b34c438b72 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:00.101459 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831eeacf-7a94-46f7-a4ad-06ed21dc6945 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed18129b40944cdda536f68868b860c1, user_id=f65a75a8587b42879cc0446fe1558225, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:00.101459 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831eeacf-7a94-46f7-a4ad-06ed21dc6945 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:00.101675 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fd4e5-b5a6-40df-875c-3e44a275990f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:00.102322 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fd4e5-b5a6-40df-875c-3e44a275990f admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:00.102620 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fd4e5-b5a6-40df-875c-3e44a275990f admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:00.102971 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fd4e5-b5a6-40df-875c-3e44a275990f admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:00.103719 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2fd4e5-b5a6-40df-875c-3e44a275990f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:00.105449 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2fd4e5-b5a6-40df-875c-3e44a275990f admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:00.123595 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 926/1865] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:00 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 12:21:00.132579 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fd4e5-b5a6-40df-875c-3e44a275990f admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:00.149177 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fd4e5-b5a6-40df-875c-3e44a275990f admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:00.156155 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381adb52-dbf2-4943-afd4-710d59f2b1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:00.156985 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381adb52-dbf2-4943-afd4-710d59f2b1ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:00.157381 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381adb52-dbf2-4943-afd4-710d59f2b1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:00.157666 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381adb52-dbf2-4943-afd4-710d59f2b1ca admin admin] PATH_INFO: `/v3/projects/82ba8d6fa93d4456b34c10362740e180/users/ce5fdef64e25456dabf8471b7d6fb0ed/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:00.186285 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 940/1866] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:21:00 2022] PUT /identity/v3/projects/ed18129b40944cdda536f68868b860c1/users/f65a75a8587b42879cc0446fe1558225/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:00.207556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642d3b5-6b64-4a24-9cf4-1bc3111a9d1b None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:00.207556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642d3b5-6b64-4a24-9cf4-1bc3111a9d1b None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:00.207556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642d3b5-6b64-4a24-9cf4-1bc3111a9d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:00.231447 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381adb52-dbf2-4943-afd4-710d59f2b1ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=82ba8d6fa93d4456b34c10362740e180, user_id=ce5fdef64e25456dabf8471b7d6fb0ed, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:00.237028 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381adb52-dbf2-4943-afd4-710d59f2b1ca admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:00.257337 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-c642d3b5-6b64-4a24-9cf4-1bc3111a9d1b None None] MFA Rules not processed for user `f65a75a8587b42879cc0446fe1558225`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:00.258965 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-c642d3b5-6b64-4a24-9cf4-1bc3111a9d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:00.378992 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 941/1867] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:00.390500 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 927/1868] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:21:00 2022] PUT /identity/v3/projects/82ba8d6fa93d4456b34c10362740e180/users/ce5fdef64e25456dabf8471b7d6fb0ed/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:00.391802 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c642d3b5-6b64-4a24-9cf4-1bc3111a9d1b None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:00.392105 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381adb52-dbf2-4943-afd4-710d59f2b1ca admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:00.554713 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381adb52-dbf2-4943-afd4-710d59f2b1ca admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:00.563127 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0dd501-e7d4-4a17-b270-81d50d4d3a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:00.563312 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c642d3b5-6b64-4a24-9cf4-1bc3111a9d1b None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:00.563926 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0dd501-e7d4-4a17-b270-81d50d4d3a93 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:00.564310 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0dd501-e7d4-4a17-b270-81d50d4d3a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:00.564833 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0dd501-e7d4-4a17-b270-81d50d4d3a93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:00.565833 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0dd501-e7d4-4a17-b270-81d50d4d3a93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:00.567719 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0dd501-e7d4-4a17-b270-81d50d4d3a93 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:00.568174 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6420178-7043-4c74-9b3a-df6fb63619b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:00.569037 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6420178-7043-4c74-9b3a-df6fb63619b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:00.569506 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6420178-7043-4c74-9b3a-df6fb63619b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:00.570258 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6420178-7043-4c74-9b3a-df6fb63619b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:00.571230 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6420178-7043-4c74-9b3a-df6fb63619b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:00.573963 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6420178-7043-4c74-9b3a-df6fb63619b5 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:00.587236 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:00.593130 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 928/1869] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:00 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:21:00.597659 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0dd501-e7d4-4a17-b270-81d50d4d3a93 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:00.611527 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0dd501-e7d4-4a17-b270-81d50d4d3a93 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:00.617659 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec3bd92-0d01-44d9-aaf7-b45d8e07d868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:00.618517 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3bd92-0d01-44d9-aaf7-b45d8e07d868 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:00.618829 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3bd92-0d01-44d9-aaf7-b45d8e07d868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:00.619108 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3bd92-0d01-44d9-aaf7-b45d8e07d868 admin admin] PATH_INFO: `/v3/projects/82ba8d6fa93d4456b34c10362740e180/users/ce5fdef64e25456dabf8471b7d6fb0ed/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:00.621780 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec3bd92-0d01-44d9-aaf7-b45d8e07d868 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82ba8d6fa93d4456b34c10362740e180, user_id=ce5fdef64e25456dabf8471b7d6fb0ed, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:00.627565 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec3bd92-0d01-44d9-aaf7-b45d8e07d868 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:00.643548 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:00.645418 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 929/1870] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:21:00 2022] PUT /identity/v3/projects/82ba8d6fa93d4456b34c10362740e180/users/ce5fdef64e25456dabf8471b7d6fb0ed/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:21:00.662957 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c06e01-7f2d-4449-86fc-00cdba6a4216 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:00.663250 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c06e01-7f2d-4449-86fc-00cdba6a4216 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:00.663609 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c06e01-7f2d-4449-86fc-00cdba6a4216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:00.707302 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 942/1871] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:21:00 2022] POST /identity/v3/projects => generated 369 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:21:00.711040 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6420178-7043-4c74-9b3a-df6fb63619b5 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:00.716417 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-57c06e01-7f2d-4449-86fc-00cdba6a4216 None None] MFA Rules not processed for user `ce5fdef64e25456dabf8471b7d6fb0ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:00.717769 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-57c06e01-7f2d-4449-86fc-00cdba6a4216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:00.804777 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 930/1872] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:00 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:00.805688 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6420178-7043-4c74-9b3a-df6fb63619b5 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:00.815551 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59b058-b4d9-47f9-8efe-8a0f90f3a219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:00.816380 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59b058-b4d9-47f9-8efe-8a0f90f3a219 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:00.816565 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59b058-b4d9-47f9-8efe-8a0f90f3a219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:00.816905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59b058-b4d9-47f9-8efe-8a0f90f3a219 admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:00.817556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b59b058-b4d9-47f9-8efe-8a0f90f3a219 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:00.819739 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b59b058-b4d9-47f9-8efe-8a0f90f3a219 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:00.822299 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c06e01-7f2d-4449-86fc-00cdba6a4216 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:00.823942 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-57c06e01-7f2d-4449-86fc-00cdba6a4216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:00.930888 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 943/1873] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:21:00 2022] POST /identity/v3/users => generated 291 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:21:00.934213 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c06e01-7f2d-4449-86fc-00cdba6a4216 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:00.937763 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59b058-b4d9-47f9-8efe-8a0f90f3a219 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:00.942473 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a69e573-2cd5-4d31-8d68-e7884559fabe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:00.949315 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69e573-2cd5-4d31-8d68-e7884559fabe service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:00.949892 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69e573-2cd5-4d31-8d68-e7884559fabe service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:00.950252 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69e573-2cd5-4d31-8d68-e7884559fabe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:00.963945 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59b058-b4d9-47f9-8efe-8a0f90f3a219 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:00.967292 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a69e573-2cd5-4d31-8d68-e7884559fabe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:00.968953 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a69e573-2cd5-4d31-8d68-e7884559fabe service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:00.969931 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb042b-deb1-4816-867c-a1b8b9d940e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:00.970558 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb042b-deb1-4816-867c-a1b8b9d940e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:00.970843 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb042b-deb1-4816-867c-a1b8b9d940e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:00.971121 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb042b-deb1-4816-867c-a1b8b9d940e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:00.971921 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fb042b-deb1-4816-867c-a1b8b9d940e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:00.974112 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fb042b-deb1-4816-867c-a1b8b9d940e8 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:00.981213 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 931/1874] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:21:00 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:21:00.995084 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 944/1875] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:00 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:21:00.995548 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a69e573-2cd5-4d31-8d68-e7884559fabe service cinder] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:01.008309 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a69e573-2cd5-4d31-8d68-e7884559fabe service cinder] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:01.013931 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eda3990-1e3d-4283-9cab-fc6e6ccbed18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:01.018985 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda3990-1e3d-4283-9cab-fc6e6ccbed18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:01.018985 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda3990-1e3d-4283-9cab-fc6e6ccbed18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:01.018985 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda3990-1e3d-4283-9cab-fc6e6ccbed18 admin admin] PATH_INFO: `/v3/projects/cf8bd977d56a457a85b3993e4ca0663d/users/d033e6753a5e402083784ac82ab502b5/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:01.034935 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eda3990-1e3d-4283-9cab-fc6e6ccbed18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf8bd977d56a457a85b3993e4ca0663d, user_id=d033e6753a5e402083784ac82ab502b5, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:01.037912 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eda3990-1e3d-4283-9cab-fc6e6ccbed18 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:01.641605 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 932/1876] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:21:00 2022] PUT /identity/v3/projects/cf8bd977d56a457a85b3993e4ca0663d/users/d033e6753a5e402083784ac82ab502b5/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 646 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:01.645627 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb042b-deb1-4816-867c-a1b8b9d940e8 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:01.657692 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb042b-deb1-4816-867c-a1b8b9d940e8 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:01.737338 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93da0e20-fc71-4bd1-b88b-46ed3975b023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:01.737983 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da0e20-fc71-4bd1-b88b-46ed3975b023 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:01.738343 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da0e20-fc71-4bd1-b88b-46ed3975b023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:01.738930 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da0e20-fc71-4bd1-b88b-46ed3975b023 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:01.739816 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93da0e20-fc71-4bd1-b88b-46ed3975b023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:01.741636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93da0e20-fc71-4bd1-b88b-46ed3975b023 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:01.760778 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 945/1877] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:01 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:21:01.762733 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eda3990-1e3d-4283-9cab-fc6e6ccbed18 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:01.774852 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eda3990-1e3d-4283-9cab-fc6e6ccbed18 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:01.780259 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a6a6f9-5d2e-401c-b8fe-a27e5fba9888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:01.780947 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a6a6f9-5d2e-401c-b8fe-a27e5fba9888 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:01.781226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a6a6f9-5d2e-401c-b8fe-a27e5fba9888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:01.781496 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a6a6f9-5d2e-401c-b8fe-a27e5fba9888 admin admin] PATH_INFO: `/v3/projects/cf8bd977d56a457a85b3993e4ca0663d/users/d033e6753a5e402083784ac82ab502b5/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:01.783554 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a6a6f9-5d2e-401c-b8fe-a27e5fba9888 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf8bd977d56a457a85b3993e4ca0663d, user_id=d033e6753a5e402083784ac82ab502b5, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:01.797339 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a6a6f9-5d2e-401c-b8fe-a27e5fba9888 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:01.797606 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 933/1878] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:21:01 2022] PUT /identity/v3/projects/cf8bd977d56a457a85b3993e4ca0663d/users/d033e6753a5e402083784ac82ab502b5/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:21:01.808391 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f44e98-6109-4ae2-a8d6-d04325066158 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:01.808684 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f44e98-6109-4ae2-a8d6-d04325066158 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:01.808965 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f44e98-6109-4ae2-a8d6-d04325066158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:01.840335 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-35f44e98-6109-4ae2-a8d6-d04325066158 None None] MFA Rules not processed for user `d033e6753a5e402083784ac82ab502b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:01.842327 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-35f44e98-6109-4ae2-a8d6-d04325066158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:01.943629 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 946/1879] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:01 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:01.955406 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a6a6f9-5d2e-401c-b8fe-a27e5fba9888 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:01.957289 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-24a6a6f9-5d2e-401c-b8fe-a27e5fba9888 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:01.963856 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f44e98-6109-4ae2-a8d6-d04325066158 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:02.021307 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f44e98-6109-4ae2-a8d6-d04325066158 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:02.022302 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a6a6f9-5d2e-401c-b8fe-a27e5fba9888 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:02.025637 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b6627-3eb1-41d3-85f8-9a941a1a4973 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:02.029700 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b6627-3eb1-41d3-85f8-9a941a1a4973 service cinder] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:02.029700 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b6627-3eb1-41d3-85f8-9a941a1a4973 service cinder] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:02.029700 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b6627-3eb1-41d3-85f8-9a941a1a4973 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:02.029975 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a2a134-e288-496f-a7f8-ef572d59af32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:02.029975 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2a134-e288-496f-a7f8-ef572d59af32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:02.029975 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2a134-e288-496f-a7f8-ef572d59af32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:02.029975 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2a134-e288-496f-a7f8-ef572d59af32 admin admin] PATH_INFO: `/v3/users/32121bd691604ef0ac646d7491aee60a` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:02.032330 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a2a134-e288-496f-a7f8-ef572d59af32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32121bd691604ef0ac646d7491aee60a)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:02.034608 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a2a134-e288-496f-a7f8-ef572d59af32 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:02.037361 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b6627-3eb1-41d3-85f8-9a941a1a4973 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:02.038902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b6627-3eb1-41d3-85f8-9a941a1a4973 service cinder] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:02.050748 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 947/1880] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:21:01 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:21:02.229636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b6627-3eb1-41d3-85f8-9a941a1a4973 service cinder] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:02.238792 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b6627-3eb1-41d3-85f8-9a941a1a4973 service cinder] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:02.247257 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47220c2-4726-4861-89e4-5670477f2024 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:02.247257 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47220c2-4726-4861-89e4-5670477f2024 service cinder] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:02.247257 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47220c2-4726-4861-89e4-5670477f2024 service cinder] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:02.247257 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47220c2-4726-4861-89e4-5670477f2024 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:02.308557 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47220c2-4726-4861-89e4-5670477f2024 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:02.309994 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47220c2-4726-4861-89e4-5670477f2024 service cinder] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:02.319523 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 948/1881] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:21:02 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:21:02.332546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-78a2a134-e288-496f-a7f8-ef572d59af32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32121bd691604ef0ac646d7491aee60a'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:02.333347 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-78a2a134-e288-496f-a7f8-ef572d59af32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32121bd691604ef0ac646d7491aee60a'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:02.334133 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-78a2a134-e288-496f-a7f8-ef572d59af32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32121bd691604ef0ac646d7491aee60a'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:02.343073 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-78a2a134-e288-496f-a7f8-ef572d59af32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32121bd691604ef0ac646d7491aee60a'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:02.356859 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-78a2a134-e288-496f-a7f8-ef572d59af32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32121bd691604ef0ac646d7491aee60a'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:02.449773 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 934/1882] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:21:01 2022] DELETE /identity/v3/users/32121bd691604ef0ac646d7491aee60a => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:02.454798 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9486701-5aca-437b-9a68-331a13bb5643 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:02.455093 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9486701-5aca-437b-9a68-331a13bb5643 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:02.455378 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9486701-5aca-437b-9a68-331a13bb5643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:02.474350 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-f9486701-5aca-437b-9a68-331a13bb5643 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:02.475615 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-f9486701-5aca-437b-9a68-331a13bb5643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:02.542928 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 949/1883] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:02.553559 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a2a134-e288-496f-a7f8-ef572d59af32 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:02.558926 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-78a2a134-e288-496f-a7f8-ef572d59af32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:02.613200 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a2a134-e288-496f-a7f8-ef572d59af32 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:02.616882 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee211b-da29-4a40-a18d-6445798e1105 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:02.617394 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee211b-da29-4a40-a18d-6445798e1105 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:02.617671 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee211b-da29-4a40-a18d-6445798e1105 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:02.617944 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee211b-da29-4a40-a18d-6445798e1105 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:02.627705 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dee211b-da29-4a40-a18d-6445798e1105 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:02.629121 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dee211b-da29-4a40-a18d-6445798e1105 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:02.638362 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 935/1884] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:21:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:21:02.931950 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9486701-5aca-437b-9a68-331a13bb5643 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:02.933450 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-f9486701-5aca-437b-9a68-331a13bb5643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:02.949153 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9486701-5aca-437b-9a68-331a13bb5643 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:02.959056 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbea1e0-ed27-4630-8750-b67692c5fcdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:02.959937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbea1e0-ed27-4630-8750-b67692c5fcdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:02.960243 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbea1e0-ed27-4630-8750-b67692c5fcdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:02.960498 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbea1e0-ed27-4630-8750-b67692c5fcdb admin admin] PATH_INFO: `/v3/users/be38f43e768046e7a539991f9af44ed0` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:02.961645 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbea1e0-ed27-4630-8750-b67692c5fcdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=be38f43e768046e7a539991f9af44ed0)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:02.963601 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbea1e0-ed27-4630-8750-b67692c5fcdb admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:03.165046 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee211b-da29-4a40-a18d-6445798e1105 service neutron] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:03.166765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-5dee211b-da29-4a40-a18d-6445798e1105 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:03.168968 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-9dbea1e0-ed27-4630-8750-b67692c5fcdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be38f43e768046e7a539991f9af44ed0'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:03.178106 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-9dbea1e0-ed27-4630-8750-b67692c5fcdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be38f43e768046e7a539991f9af44ed0'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:03.179095 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-9dbea1e0-ed27-4630-8750-b67692c5fcdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be38f43e768046e7a539991f9af44ed0'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:03.181790 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee211b-da29-4a40-a18d-6445798e1105 service neutron] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:03.187997 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00006f4-da4e-43f3-a6ad-0e9915fd9130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:03.190855 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00006f4-da4e-43f3-a6ad-0e9915fd9130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:03.190855 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00006f4-da4e-43f3-a6ad-0e9915fd9130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:03.190855 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00006f4-da4e-43f3-a6ad-0e9915fd9130 admin admin] PATH_INFO: `/v3/projects/87abee0f1d844fa1a5e10a1b6690ab0c` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:03.190855 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00006f4-da4e-43f3-a6ad-0e9915fd9130 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87abee0f1d844fa1a5e10a1b6690ab0c)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:03.192471 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00006f4-da4e-43f3-a6ad-0e9915fd9130 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:03.192937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-9dbea1e0-ed27-4630-8750-b67692c5fcdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be38f43e768046e7a539991f9af44ed0'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:03.270584 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-9dbea1e0-ed27-4630-8750-b67692c5fcdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be38f43e768046e7a539991f9af44ed0'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:03.271586 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:03.282162 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 950/1885] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:21:02 2022] DELETE /identity/v3/users/be38f43e768046e7a539991f9af44ed0 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:03.285500 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33cf1a7-b0f9-4e81-adfa-b01b168ef612 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:03.285979 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33cf1a7-b0f9-4e81-adfa-b01b168ef612 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:03.286577 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33cf1a7-b0f9-4e81-adfa-b01b168ef612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:03.314495 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-e33cf1a7-b0f9-4e81-adfa-b01b168ef612 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:03.315683 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-e33cf1a7-b0f9-4e81-adfa-b01b168ef612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:03.366761 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-c00006f4-da4e-43f3-a6ad-0e9915fd9130 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87abee0f1d844fa1a5e10a1b6690ab0c', 'request_id': 'req-c00006f4-da4e-43f3-a6ad-0e9915fd9130'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:03.369927 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-c00006f4-da4e-43f3-a6ad-0e9915fd9130 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87abee0f1d844fa1a5e10a1b6690ab0c', 'request_id': 'req-c00006f4-da4e-43f3-a6ad-0e9915fd9130'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:03.392383 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:03.413793 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 951/1886] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:03.423834 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33cf1a7-b0f9-4e81-adfa-b01b168ef612 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:03.425321 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-e33cf1a7-b0f9-4e81-adfa-b01b168ef612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:03.460235 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-c00006f4-da4e-43f3-a6ad-0e9915fd9130 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87abee0f1d844fa1a5e10a1b6690ab0c', 'request_id': 'req-c00006f4-da4e-43f3-a6ad-0e9915fd9130'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:03.475870 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-c00006f4-da4e-43f3-a6ad-0e9915fd9130 admin admin] The token cache is being invalidate because project 87abee0f1d844fa1a5e10a1b6690ab0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:21:03.476621 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-c00006f4-da4e-43f3-a6ad-0e9915fd9130 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:03.477858 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-c00006f4-da4e-43f3-a6ad-0e9915fd9130 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:03.479384 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 936/1887] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:21:03 2022] DELETE /identity/v3/projects/87abee0f1d844fa1a5e10a1b6690ab0c => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:21:03.505508 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd295c-436f-401f-9b93-c2aa7cbe22b1 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:03.505508 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd295c-436f-401f-9b93-c2aa7cbe22b1 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:03.505508 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd295c-436f-401f-9b93-c2aa7cbe22b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:03.506093 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33cf1a7-b0f9-4e81-adfa-b01b168ef612 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:03.509192 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-b5fd295c-436f-401f-9b93-c2aa7cbe22b1 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:03.510421 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-b5fd295c-436f-401f-9b93-c2aa7cbe22b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:03.549065 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c6b5a-e62a-4f16-a4d6-09f08f31d40d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:03.549781 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c6b5a-e62a-4f16-a4d6-09f08f31d40d service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:03.550071 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c6b5a-e62a-4f16-a4d6-09f08f31d40d service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:03.550353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c6b5a-e62a-4f16-a4d6-09f08f31d40d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:03.552151 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-618c6b5a-e62a-4f16-a4d6-09f08f31d40d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:03.573438 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 937/1888] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:21:03.581828 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd023501-6db9-4da2-8cae-afbfb5f680e6 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:03.582099 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd023501-6db9-4da2-8cae-afbfb5f680e6 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:03.582386 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd023501-6db9-4da2-8cae-afbfb5f680e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:03.602135 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-fd023501-6db9-4da2-8cae-afbfb5f680e6 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:03.604928 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-fd023501-6db9-4da2-8cae-afbfb5f680e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:03.611397 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 938/1889] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:21:03.615712 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd023501-6db9-4da2-8cae-afbfb5f680e6 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:03.620587 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618c6b5a-e62a-4f16-a4d6-09f08f31d40d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:03.622289 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618c6b5a-e62a-4f16-a4d6-09f08f31d40d service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:03.625666 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd023501-6db9-4da2-8cae-afbfb5f680e6 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:03.630616 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6f198-da42-477b-98df-c644ca4ec840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:03.631218 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6f198-da42-477b-98df-c644ca4ec840 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:03.631540 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6f198-da42-477b-98df-c644ca4ec840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:03.631822 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6f198-da42-477b-98df-c644ca4ec840 admin admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:03.632481 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd6f198-da42-477b-98df-c644ca4ec840 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:21:03.632615 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 952/1890] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:21:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:21:03.632794 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd6f198-da42-477b-98df-c644ca4ec840 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:03.634395 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd6f198-da42-477b-98df-c644ca4ec840 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:03.647598 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 939/1891] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:21:03.656212 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c6b5a-e62a-4f16-a4d6-09f08f31d40d service neutron] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:03.666960 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c6b5a-e62a-4f16-a4d6-09f08f31d40d service neutron] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:03.679478 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66792082-abb1-4923-aa59-155c6196955c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:03.680727 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66792082-abb1-4923-aa59-155c6196955c admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:03.681183 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66792082-abb1-4923-aa59-155c6196955c admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:03.681550 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66792082-abb1-4923-aa59-155c6196955c admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:03.682863 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66792082-abb1-4923-aa59-155c6196955c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:03.685606 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66792082-abb1-4923-aa59-155c6196955c admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:03.805228 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 953/1892] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:21:03 2022] POST /identity/v3/projects => generated 369 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:21:03.813713 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6f198-da42-477b-98df-c644ca4ec840 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:03.823789 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6f198-da42-477b-98df-c644ca4ec840 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:03.885219 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c20b56-a062-4c40-a2cf-2efc72633671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:03.885844 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c20b56-a062-4c40-a2cf-2efc72633671 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:03.886126 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c20b56-a062-4c40-a2cf-2efc72633671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:03.886532 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c20b56-a062-4c40-a2cf-2efc72633671 admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:03.887136 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c20b56-a062-4c40-a2cf-2efc72633671 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:03.889724 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c20b56-a062-4c40-a2cf-2efc72633671 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:04.020214 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 940/1893] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:21:03 2022] POST /identity/v3/users => generated 285 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:21:04.024385 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66792082-abb1-4923-aa59-155c6196955c admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:04.035330 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66792082-abb1-4923-aa59-155c6196955c admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:04.040148 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f888424-7c2c-44f2-ae34-596a045fbb32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:04.040835 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f888424-7c2c-44f2-ae34-596a045fbb32 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:04.041163 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f888424-7c2c-44f2-ae34-596a045fbb32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:04.041534 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f888424-7c2c-44f2-ae34-596a045fbb32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:04.042262 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f888424-7c2c-44f2-ae34-596a045fbb32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:04.044012 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f888424-7c2c-44f2-ae34-596a045fbb32 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:04.055816 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 954/1894] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:21:04.059856 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c20b56-a062-4c40-a2cf-2efc72633671 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:04.071240 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c20b56-a062-4c40-a2cf-2efc72633671 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:04.076496 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687f41c-b1f6-4d1c-86b3-8ca4853635da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:04.080809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687f41c-b1f6-4d1c-86b3-8ca4853635da admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:04.080809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687f41c-b1f6-4d1c-86b3-8ca4853635da admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:04.080809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687f41c-b1f6-4d1c-86b3-8ca4853635da admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:04.080809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687f41c-b1f6-4d1c-86b3-8ca4853635da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:04.080809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687f41c-b1f6-4d1c-86b3-8ca4853635da admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:04.101387 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 941/1895] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 12:21:04.105623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f888424-7c2c-44f2-ae34-596a045fbb32 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:04.121049 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f888424-7c2c-44f2-ae34-596a045fbb32 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:04.122292 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9035a9-4bbc-435e-819f-a53d422cc3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:04.123651 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9035a9-4bbc-435e-819f-a53d422cc3ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:04.123933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9035a9-4bbc-435e-819f-a53d422cc3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:04.124220 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9035a9-4bbc-435e-819f-a53d422cc3ce admin admin] PATH_INFO: `/v3/projects/edf75c0cdb814a1cb74e33660e16429c/users/2ac6813dae8b4496a8a5f5ef98407530/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:04.125116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687f41c-b1f6-4d1c-86b3-8ca4853635da admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:04.127034 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-b687f41c-b1f6-4d1c-86b3-8ca4853635da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:04.143927 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9035a9-4bbc-435e-819f-a53d422cc3ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=edf75c0cdb814a1cb74e33660e16429c, user_id=2ac6813dae8b4496a8a5f5ef98407530, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:04.146785 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9035a9-4bbc-435e-819f-a53d422cc3ce admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:04.152601 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687f41c-b1f6-4d1c-86b3-8ca4853635da admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:04.158694 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998bce78-9b14-4a37-b951-685a328869c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:04.159714 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998bce78-9b14-4a37-b951-685a328869c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:04.160150 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998bce78-9b14-4a37-b951-685a328869c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:04.160616 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998bce78-9b14-4a37-b951-685a328869c9 admin admin] PATH_INFO: `/v3/projects/dbb4fe406c5f4d8ba0494e16688b263a` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:04.161945 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998bce78-9b14-4a37-b951-685a328869c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbb4fe406c5f4d8ba0494e16688b263a)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:04.163988 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998bce78-9b14-4a37-b951-685a328869c9 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:04.231851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 955/1896] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:21:04 2022] PUT /identity/v3/projects/edf75c0cdb814a1cb74e33660e16429c/users/2ac6813dae8b4496a8a5f5ef98407530/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:04.242564 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f5fda-cc88-4bb3-a68c-8351508c3f2f None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:04.242930 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f5fda-cc88-4bb3-a68c-8351508c3f2f None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:04.243285 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f5fda-cc88-4bb3-a68c-8351508c3f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:04.272527 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-327f5fda-cc88-4bb3-a68c-8351508c3f2f None None] MFA Rules not processed for user `2ac6813dae8b4496a8a5f5ef98407530`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:04.273786 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-327f5fda-cc88-4bb3-a68c-8351508c3f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:04.300598 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-998bce78-9b14-4a37-b951-685a328869c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbb4fe406c5f4d8ba0494e16688b263a', 'request_id': 'req-998bce78-9b14-4a37-b951-685a328869c9'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:04.306873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-998bce78-9b14-4a37-b951-685a328869c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbb4fe406c5f4d8ba0494e16688b263a', 'request_id': 'req-998bce78-9b14-4a37-b951-685a328869c9'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:04.350294 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 956/1897] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:21:04.368470 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f5fda-cc88-4bb3-a68c-8351508c3f2f None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:04.370003 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-327f5fda-cc88-4bb3-a68c-8351508c3f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:04.418847 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-998bce78-9b14-4a37-b951-685a328869c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbb4fe406c5f4d8ba0494e16688b263a', 'request_id': 'req-998bce78-9b14-4a37-b951-685a328869c9'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:04.434468 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-998bce78-9b14-4a37-b951-685a328869c9 admin admin] The token cache is being invalidate because project dbb4fe406c5f4d8ba0494e16688b263a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:21:04.435013 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-998bce78-9b14-4a37-b951-685a328869c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:04.436023 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-998bce78-9b14-4a37-b951-685a328869c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:04.439053 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 942/1898] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:21:04 2022] DELETE /identity/v3/projects/dbb4fe406c5f4d8ba0494e16688b263a => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:04.460760 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998bce78-9b14-4a37-b951-685a328869c9 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:04.463685 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-998bce78-9b14-4a37-b951-685a328869c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:04.481302 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f5fda-cc88-4bb3-a68c-8351508c3f2f None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:04.543981 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cdf37b-f36a-45ad-aec1-2c090617da02 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd020063702b0481e9f4111680766f7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:04.546135 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdf37b-f36a-45ad-aec1-2c090617da02 service glance] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:04.546135 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdf37b-f36a-45ad-aec1-2c090617da02 service glance] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:04.546135 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdf37b-f36a-45ad-aec1-2c090617da02 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:04.547620 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-77cdf37b-f36a-45ad-aec1-2c090617da02 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:04.555108 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998bce78-9b14-4a37-b951-685a328869c9 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:04.560366 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa13be-e972-437c-a211-6264701d7c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:04.561375 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa13be-e972-437c-a211-6264701d7c2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:04.561734 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa13be-e972-437c-a211-6264701d7c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:04.562087 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa13be-e972-437c-a211-6264701d7c2f admin admin] PATH_INFO: `/v3/users/ce5fdef64e25456dabf8471b7d6fb0ed` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:04.563359 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa13be-e972-437c-a211-6264701d7c2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce5fdef64e25456dabf8471b7d6fb0ed)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:04.565394 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa13be-e972-437c-a211-6264701d7c2f admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:04.619854 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cdf37b-f36a-45ad-aec1-2c090617da02 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:04.620794 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cdf37b-f36a-45ad-aec1-2c090617da02 service glance] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:04.633025 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 957/1899] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:21:04 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:21:04.823046 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:04.837668 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-dcfa13be-e972-437c-a211-6264701d7c2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce5fdef64e25456dabf8471b7d6fb0ed'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:04.840874 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-dcfa13be-e972-437c-a211-6264701d7c2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce5fdef64e25456dabf8471b7d6fb0ed'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:04.841507 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-dcfa13be-e972-437c-a211-6264701d7c2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce5fdef64e25456dabf8471b7d6fb0ed'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:04.851384 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-dcfa13be-e972-437c-a211-6264701d7c2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce5fdef64e25456dabf8471b7d6fb0ed'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:04.869155 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-dcfa13be-e972-437c-a211-6264701d7c2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce5fdef64e25456dabf8471b7d6fb0ed'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:04.955812 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 943/1900] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:21:04 2022] DELETE /identity/v3/users/ce5fdef64e25456dabf8471b7d6fb0ed => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:21:05.312044 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:05.551578 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cdf37b-f36a-45ad-aec1-2c090617da02 service glance] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:05.563229 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cdf37b-f36a-45ad-aec1-2c090617da02 service glance] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:05.624922 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8641ab-4d99-42ca-b4aa-f7cd241bd3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:05.625570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8641ab-4d99-42ca-b4aa-f7cd241bd3ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:05.625860 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8641ab-4d99-42ca-b4aa-f7cd241bd3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:05.626141 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8641ab-4d99-42ca-b4aa-f7cd241bd3ea admin admin] PATH_INFO: `/v3/projects/82ba8d6fa93d4456b34c10362740e180` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:05.627378 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8641ab-4d99-42ca-b4aa-f7cd241bd3ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=82ba8d6fa93d4456b34c10362740e180)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:05.629275 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8641ab-4d99-42ca-b4aa-f7cd241bd3ea admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:05.768300 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-4c8641ab-4d99-42ca-b4aa-f7cd241bd3ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82ba8d6fa93d4456b34c10362740e180', 'request_id': 'req-4c8641ab-4d99-42ca-b4aa-f7cd241bd3ea'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:05.774068 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-4c8641ab-4d99-42ca-b4aa-f7cd241bd3ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82ba8d6fa93d4456b34c10362740e180', 'request_id': 'req-4c8641ab-4d99-42ca-b4aa-f7cd241bd3ea'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:05.837116 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-4c8641ab-4d99-42ca-b4aa-f7cd241bd3ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82ba8d6fa93d4456b34c10362740e180', 'request_id': 'req-4c8641ab-4d99-42ca-b4aa-f7cd241bd3ea'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:05.851709 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-4c8641ab-4d99-42ca-b4aa-f7cd241bd3ea admin admin] The token cache is being invalidate because project 82ba8d6fa93d4456b34c10362740e180 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:21:05.852194 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-4c8641ab-4d99-42ca-b4aa-f7cd241bd3ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:05.854200 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-4c8641ab-4d99-42ca-b4aa-f7cd241bd3ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:05.855815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 958/1901] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:21:05 2022] DELETE /identity/v3/projects/82ba8d6fa93d4456b34c10362740e180 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:05.860763 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df5807-9085-497b-bff6-a2c063c65fcb None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:05.864612 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df5807-9085-497b-bff6-a2c063c65fcb None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:05.864612 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df5807-9085-497b-bff6-a2c063c65fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:05.884137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-e5df5807-9085-497b-bff6-a2c063c65fcb None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:05.885396 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-e5df5807-9085-497b-bff6-a2c063c65fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:05.949723 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 944/1902] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:21:05.957987 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0dcea8-f90f-4d47-a3f4-c98249915864 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:05.958492 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0dcea8-f90f-4d47-a3f4-c98249915864 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:05.959452 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0dcea8-f90f-4d47-a3f4-c98249915864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:05.982264 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-0b0dcea8-f90f-4d47-a3f4-c98249915864 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:05.983537 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-0b0dcea8-f90f-4d47-a3f4-c98249915864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:05.991456 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 959/1903] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:21:05.995976 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5df5807-9085-497b-bff6-a2c063c65fcb None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:06.006851 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5df5807-9085-497b-bff6-a2c063c65fcb None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:06.011751 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0ba4f-c434-455f-ac45-a0253877ad42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:06.012398 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0ba4f-c434-455f-ac45-a0253877ad42 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:06.012737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0ba4f-c434-455f-ac45-a0253877ad42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:06.013020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0ba4f-c434-455f-ac45-a0253877ad42 admin admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:06.014335 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d0ba4f-c434-455f-ac45-a0253877ad42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:21:06.014640 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d0ba4f-c434-455f-ac45-a0253877ad42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:06.016285 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d0ba4f-c434-455f-ac45-a0253877ad42 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:06.028938 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 945/1904] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:21:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:21:06.033380 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0dcea8-f90f-4d47-a3f4-c98249915864 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:06.043053 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0dcea8-f90f-4d47-a3f4-c98249915864 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:06.048049 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3137dbc-b87f-4291-9dba-a194adef64e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:06.048662 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3137dbc-b87f-4291-9dba-a194adef64e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:06.048947 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3137dbc-b87f-4291-9dba-a194adef64e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:06.051329 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3137dbc-b87f-4291-9dba-a194adef64e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:06.051329 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3137dbc-b87f-4291-9dba-a194adef64e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:06.051838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3137dbc-b87f-4291-9dba-a194adef64e6 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:06.118801 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:06.175891 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 960/1905] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:21:06 2022] POST /identity/v3/projects => generated 359 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:21:06.180296 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0ba4f-c434-455f-ac45-a0253877ad42 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:06.190678 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0ba4f-c434-455f-ac45-a0253877ad42 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:06.247266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bbc6ec-56b5-49af-8048-a329f6772182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:06.248458 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbc6ec-56b5-49af-8048-a329f6772182 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:06.248737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbc6ec-56b5-49af-8048-a329f6772182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:06.249022 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbc6ec-56b5-49af-8048-a329f6772182 admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:06.249651 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bbc6ec-56b5-49af-8048-a329f6772182 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:06.251997 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bbc6ec-56b5-49af-8048-a329f6772182 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:06.328914 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 946/1906] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:21:06 2022] POST /identity/v3/users => generated 280 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:21:06.332339 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3137dbc-b87f-4291-9dba-a194adef64e6 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:06.353745 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3137dbc-b87f-4291-9dba-a194adef64e6 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:06.359473 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46fb015-184d-4d38-8905-04e64ec45795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:06.360045 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fb015-184d-4d38-8905-04e64ec45795 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:06.360317 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fb015-184d-4d38-8905-04e64ec45795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:06.360594 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fb015-184d-4d38-8905-04e64ec45795 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:06.361246 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46fb015-184d-4d38-8905-04e64ec45795 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:06.362842 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46fb015-184d-4d38-8905-04e64ec45795 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:06.376400 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 961/1907] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:21:06.380526 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bbc6ec-56b5-49af-8048-a329f6772182 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:06.391567 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bbc6ec-56b5-49af-8048-a329f6772182 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:06.396489 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3989598f-c1fb-4790-a828-0cd9b5c5b369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:06.397101 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3989598f-c1fb-4790-a828-0cd9b5c5b369 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:06.397423 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3989598f-c1fb-4790-a828-0cd9b5c5b369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:06.397720 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3989598f-c1fb-4790-a828-0cd9b5c5b369 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:06.398383 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3989598f-c1fb-4790-a828-0cd9b5c5b369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:06.400148 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3989598f-c1fb-4790-a828-0cd9b5c5b369 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:06.412161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 947/1908] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:21:06.416573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46fb015-184d-4d38-8905-04e64ec45795 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:06.436670 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46fb015-184d-4d38-8905-04e64ec45795 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:06.436670 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132c79c-e0e7-4d51-a421-7b299a0f8737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:06.436670 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132c79c-e0e7-4d51-a421-7b299a0f8737 admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:06.436670 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132c79c-e0e7-4d51-a421-7b299a0f8737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:06.437764 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132c79c-e0e7-4d51-a421-7b299a0f8737 admin admin] PATH_INFO: `/v3/projects/4b43ac7010384640857a94810ea6589b/users/ce57be1f2bef42228328aa73699f298c/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:06.454521 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132c79c-e0e7-4d51-a421-7b299a0f8737 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b43ac7010384640857a94810ea6589b, user_id=ce57be1f2bef42228328aa73699f298c, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:06.455872 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132c79c-e0e7-4d51-a421-7b299a0f8737 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:07.459127 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:07.466708 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 962/1909] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:21:06 2022] PUT /identity/v3/projects/4b43ac7010384640857a94810ea6589b/users/ce57be1f2bef42228328aa73699f298c/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 1050 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:21:07.478132 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596f9f9-c8e3-4e2d-8450-dbfc46f52a52 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:07.478511 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596f9f9-c8e3-4e2d-8450-dbfc46f52a52 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:07.478888 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596f9f9-c8e3-4e2d-8450-dbfc46f52a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:07.507968 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-0596f9f9-c8e3-4e2d-8450-dbfc46f52a52 None None] MFA Rules not processed for user `ce57be1f2bef42228328aa73699f298c`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:07.509661 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-0596f9f9-c8e3-4e2d-8450-dbfc46f52a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:07.634840 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 948/1910] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:07 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:07.794689 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656071be-6d14-4c66-97ff-b95bf128fab9 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:07.799266 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656071be-6d14-4c66-97ff-b95bf128fab9 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:07.799466 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656071be-6d14-4c66-97ff-b95bf128fab9 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:07.806693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 963/1911] 10.222.0.98 () {54 vars in 921 bytes} [Thu Aug 18 12:21:07 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:21:07.826146 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0646f-9845-4e64-82e7-78c0ecdb1558 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:07.826812 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0646f-9845-4e64-82e7-78c0ecdb1558 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:07.827242 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0646f-9845-4e64-82e7-78c0ecdb1558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:07.861511 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-37a0646f-9845-4e64-82e7-78c0ecdb1558 None None] MFA Rules not processed for user `171c72a6b29e418bbb4ea5c6faffe3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:07.862956 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-37a0646f-9845-4e64-82e7-78c0ecdb1558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:07.882171 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656071be-6d14-4c66-97ff-b95bf128fab9 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:07.884475 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-656071be-6d14-4c66-97ff-b95bf128fab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:07.980647 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656071be-6d14-4c66-97ff-b95bf128fab9 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:07.988621 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 949/1912] 10.222.0.98 () {58 vars in 1018 bytes} [Thu Aug 18 12:21:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:07.989441 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32445cf-a2f0-4563-892d-9a779c77c17b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:07.990299 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32445cf-a2f0-4563-892d-9a779c77c17b service cinder] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:07.990589 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32445cf-a2f0-4563-892d-9a779c77c17b service cinder] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:07.990871 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32445cf-a2f0-4563-892d-9a779c77c17b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:08.007310 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32445cf-a2f0-4563-892d-9a779c77c17b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:08.007310 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32445cf-a2f0-4563-892d-9a779c77c17b service cinder] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:08.016988 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a0646f-9845-4e64-82e7-78c0ecdb1558 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:08.018476 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-37a0646f-9845-4e64-82e7-78c0ecdb1558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:08.036133 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 964/1913] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:21:07 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:21:08.046249 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32445cf-a2f0-4563-892d-9a779c77c17b service cinder] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:08.047043 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a0646f-9845-4e64-82e7-78c0ecdb1558 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:08.052969 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7fac22-e472-46af-b50c-fc5168439622 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:08.053708 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fac22-e472-46af-b50c-fc5168439622 service nova] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:08.053945 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fac22-e472-46af-b50c-fc5168439622 service nova] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:08.054226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fac22-e472-46af-b50c-fc5168439622 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:08.067126 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7fac22-e472-46af-b50c-fc5168439622 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:08.068958 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7fac22-e472-46af-b50c-fc5168439622 service nova] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:08.083299 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 950/1914] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:21:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:21:08.148608 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32445cf-a2f0-4563-892d-9a779c77c17b service cinder] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:08.154687 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f2480-9daa-43a2-912d-44c9ed41f52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:08.155366 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f2480-9daa-43a2-912d-44c9ed41f52e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:08.155771 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f2480-9daa-43a2-912d-44c9ed41f52e admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:08.156053 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f2480-9daa-43a2-912d-44c9ed41f52e admin admin] PATH_INFO: `/v3/users/ce57be1f2bef42228328aa73699f298c` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:08.160315 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f2480-9daa-43a2-912d-44c9ed41f52e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce57be1f2bef42228328aa73699f298c)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:08.160315 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f2480-9daa-43a2-912d-44c9ed41f52e admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:08.274459 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7fac22-e472-46af-b50c-fc5168439622 service nova] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:08.276382 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-bb7fac22-e472-46af-b50c-fc5168439622 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:08.293662 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7fac22-e472-46af-b50c-fc5168439622 service nova] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:08.299242 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07fae57-6600-433e-a9c4-e2fc776e1793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:08.300158 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07fae57-6600-433e-a9c4-e2fc776e1793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:08.300581 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07fae57-6600-433e-a9c4-e2fc776e1793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:08.300864 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07fae57-6600-433e-a9c4-e2fc776e1793 admin admin] PATH_INFO: `/v3/users/fb3b39af31d8480bac0ee3a8cfe60b43` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:08.302025 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07fae57-6600-433e-a9c4-e2fc776e1793 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb3b39af31d8480bac0ee3a8cfe60b43)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:08.304082 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07fae57-6600-433e-a9c4-e2fc776e1793 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:08.333028 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:08.433488 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-dc0f2480-9daa-43a2-912d-44c9ed41f52e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce57be1f2bef42228328aa73699f298c'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:08.441042 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-dc0f2480-9daa-43a2-912d-44c9ed41f52e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce57be1f2bef42228328aa73699f298c'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:08.441979 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-dc0f2480-9daa-43a2-912d-44c9ed41f52e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce57be1f2bef42228328aa73699f298c'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:08.457410 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-dc0f2480-9daa-43a2-912d-44c9ed41f52e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce57be1f2bef42228328aa73699f298c'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:08.522782 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-dc0f2480-9daa-43a2-912d-44c9ed41f52e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce57be1f2bef42228328aa73699f298c'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:08.529541 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 965/1915] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:21:08 2022] DELETE /identity/v3/users/ce57be1f2bef42228328aa73699f298c => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:21:08.535584 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc64fe57-ef50-4e0f-9d10-404a7a167d30 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:08.535759 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc64fe57-ef50-4e0f-9d10-404a7a167d30 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:08.536032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc64fe57-ef50-4e0f-9d10-404a7a167d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:08.540385 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-d07fae57-6600-433e-a9c4-e2fc776e1793 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb3b39af31d8480bac0ee3a8cfe60b43'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:08.541074 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-d07fae57-6600-433e-a9c4-e2fc776e1793 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb3b39af31d8480bac0ee3a8cfe60b43'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:08.541698 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-d07fae57-6600-433e-a9c4-e2fc776e1793 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb3b39af31d8480bac0ee3a8cfe60b43'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:08.549422 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-d07fae57-6600-433e-a9c4-e2fc776e1793 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb3b39af31d8480bac0ee3a8cfe60b43'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:08.572520 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-dc64fe57-ef50-4e0f-9d10-404a7a167d30 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:08.572520 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-dc64fe57-ef50-4e0f-9d10-404a7a167d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:08.573080 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-d07fae57-6600-433e-a9c4-e2fc776e1793 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb3b39af31d8480bac0ee3a8cfe60b43'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:08.631587 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 966/1916] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:08.641683 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc64fe57-ef50-4e0f-9d10-404a7a167d30 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:08.643065 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-dc64fe57-ef50-4e0f-9d10-404a7a167d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:08.705133 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc64fe57-ef50-4e0f-9d10-404a7a167d30 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:08.710288 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9655ac7b-69d3-4639-abf0-8dee90982013 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:08.710288 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655ac7b-69d3-4639-abf0-8dee90982013 service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:08.710288 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655ac7b-69d3-4639-abf0-8dee90982013 service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:08.710288 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655ac7b-69d3-4639-abf0-8dee90982013 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:08.717802 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9655ac7b-69d3-4639-abf0-8dee90982013 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:08.719248 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9655ac7b-69d3-4639-abf0-8dee90982013 service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:08.729395 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 967/1917] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:21:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:21:08.744272 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 951/1918] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:21:08 2022] DELETE /identity/v3/users/fb3b39af31d8480bac0ee3a8cfe60b43 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:08.746560 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328bafb5-40c7-4c96-be9d-8d1ce6da22dc None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:08.747001 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328bafb5-40c7-4c96-be9d-8d1ce6da22dc None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:08.747102 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328bafb5-40c7-4c96-be9d-8d1ce6da22dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:08.770707 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-328bafb5-40c7-4c96-be9d-8d1ce6da22dc None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:08.771987 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-328bafb5-40c7-4c96-be9d-8d1ce6da22dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:08.841696 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 968/1919] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:08.853125 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07fae57-6600-433e-a9c4-e2fc776e1793 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:08.853125 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-d07fae57-6600-433e-a9c4-e2fc776e1793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:08.921305 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07fae57-6600-433e-a9c4-e2fc776e1793 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:08.925016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901e9c24-f341-4973-a96f-401538ffc903 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:08.925632 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e9c24-f341-4973-a96f-401538ffc903 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:08.925922 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e9c24-f341-4973-a96f-401538ffc903 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:08.926199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e9c24-f341-4973-a96f-401538ffc903 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:08.935756 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901e9c24-f341-4973-a96f-401538ffc903 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:08.937197 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901e9c24-f341-4973-a96f-401538ffc903 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:08.957494 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 952/1920] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:21:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:21:09.249761 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328bafb5-40c7-4c96-be9d-8d1ce6da22dc None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:09.251601 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-328bafb5-40c7-4c96-be9d-8d1ce6da22dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:09.266958 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328bafb5-40c7-4c96-be9d-8d1ce6da22dc None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:09.272445 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126cfa1c-3e4f-46c6-a1c5-77d094cee155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:09.273115 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126cfa1c-3e4f-46c6-a1c5-77d094cee155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:09.273390 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126cfa1c-3e4f-46c6-a1c5-77d094cee155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:09.274224 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126cfa1c-3e4f-46c6-a1c5-77d094cee155 admin admin] PATH_INFO: `/v3/projects/4b43ac7010384640857a94810ea6589b` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:09.275105 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126cfa1c-3e4f-46c6-a1c5-77d094cee155 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b43ac7010384640857a94810ea6589b)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:09.277152 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126cfa1c-3e4f-46c6-a1c5-77d094cee155 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:09.323363 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:09.364213 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901e9c24-f341-4973-a96f-401538ffc903 service neutron] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:09.365814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-901e9c24-f341-4973-a96f-401538ffc903 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:09.380134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901e9c24-f341-4973-a96f-401538ffc903 service neutron] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:09.386752 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ce9657-03d3-49d3-bb5b-c1341e1a1e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:09.387794 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce9657-03d3-49d3-bb5b-c1341e1a1e27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:09.388817 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce9657-03d3-49d3-bb5b-c1341e1a1e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:09.389384 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce9657-03d3-49d3-bb5b-c1341e1a1e27 admin admin] PATH_INFO: `/v3/users/f65a75a8587b42879cc0446fe1558225` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:09.390783 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ce9657-03d3-49d3-bb5b-c1341e1a1e27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f65a75a8587b42879cc0446fe1558225)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:09.392956 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ce9657-03d3-49d3-bb5b-c1341e1a1e27 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:09.448123 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-126cfa1c-3e4f-46c6-a1c5-77d094cee155 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b43ac7010384640857a94810ea6589b', 'request_id': 'req-126cfa1c-3e4f-46c6-a1c5-77d094cee155'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:09.454339 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-126cfa1c-3e4f-46c6-a1c5-77d094cee155 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b43ac7010384640857a94810ea6589b', 'request_id': 'req-126cfa1c-3e4f-46c6-a1c5-77d094cee155'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:09.459222 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:09.580527 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-126cfa1c-3e4f-46c6-a1c5-77d094cee155 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b43ac7010384640857a94810ea6589b', 'request_id': 'req-126cfa1c-3e4f-46c6-a1c5-77d094cee155'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:09.593370 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-126cfa1c-3e4f-46c6-a1c5-77d094cee155 admin admin] The token cache is being invalidate because project 4b43ac7010384640857a94810ea6589b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:21:09.593662 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-126cfa1c-3e4f-46c6-a1c5-77d094cee155 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:09.594318 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-126cfa1c-3e4f-46c6-a1c5-77d094cee155 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:09.596116 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 969/1921] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:21:09 2022] DELETE /identity/v3/projects/4b43ac7010384640857a94810ea6589b => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:09.596875 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126cfa1c-3e4f-46c6-a1c5-77d094cee155 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:09.598269 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-126cfa1c-3e4f-46c6-a1c5-77d094cee155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:09.662108 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126cfa1c-3e4f-46c6-a1c5-77d094cee155 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:09.667292 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ff877-29c0-47c7-b769-6229e124701d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:09.667970 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ff877-29c0-47c7-b769-6229e124701d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:09.668288 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ff877-29c0-47c7-b769-6229e124701d admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:09.668573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ff877-29c0-47c7-b769-6229e124701d admin admin] PATH_INFO: `/v3/projects/55524dfedf924164a1d57f8b236c2afd` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:09.669718 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ff877-29c0-47c7-b769-6229e124701d admin admin] RBAC: Authorizing `identity:delete_project(project_id=55524dfedf924164a1d57f8b236c2afd)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:09.671685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ff877-29c0-47c7-b769-6229e124701d admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:09.715034 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-e6ce9657-03d3-49d3-bb5b-c1341e1a1e27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f65a75a8587b42879cc0446fe1558225'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:09.715903 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-e6ce9657-03d3-49d3-bb5b-c1341e1a1e27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f65a75a8587b42879cc0446fe1558225'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:09.716556 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-e6ce9657-03d3-49d3-bb5b-c1341e1a1e27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f65a75a8587b42879cc0446fe1558225'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:09.723385 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-e6ce9657-03d3-49d3-bb5b-c1341e1a1e27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f65a75a8587b42879cc0446fe1558225'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:09.734959 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-e6ce9657-03d3-49d3-bb5b-c1341e1a1e27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f65a75a8587b42879cc0446fe1558225'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:09.780522 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-158ff877-29c0-47c7-b769-6229e124701d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55524dfedf924164a1d57f8b236c2afd', 'request_id': 'req-158ff877-29c0-47c7-b769-6229e124701d'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:09.786097 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-158ff877-29c0-47c7-b769-6229e124701d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55524dfedf924164a1d57f8b236c2afd', 'request_id': 'req-158ff877-29c0-47c7-b769-6229e124701d'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:09.826564 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 953/1922] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:21:09 2022] DELETE /identity/v3/users/f65a75a8587b42879cc0446fe1558225 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:21:09.827950 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a0c20-22ca-4b89-8043-b0d3a3113795 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:09.832644 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a0c20-22ca-4b89-8043-b0d3a3113795 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:09.832644 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a0c20-22ca-4b89-8043-b0d3a3113795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:09.849545 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-1d5a0c20-22ca-4b89-8043-b0d3a3113795 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:09.850728 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-1d5a0c20-22ca-4b89-8043-b0d3a3113795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:09.913437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 954/1923] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:09.914786 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e16ee5-7d20-49dd-b7f8-3ca6cb740c19 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:09.915035 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e16ee5-7d20-49dd-b7f8-3ca6cb740c19 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:09.915464 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e16ee5-7d20-49dd-b7f8-3ca6cb740c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:09.927927 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-158ff877-29c0-47c7-b769-6229e124701d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55524dfedf924164a1d57f8b236c2afd', 'request_id': 'req-158ff877-29c0-47c7-b769-6229e124701d'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:09.934565 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-05e16ee5-7d20-49dd-b7f8-3ca6cb740c19 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:09.936349 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-05e16ee5-7d20-49dd-b7f8-3ca6cb740c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:09.940123 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-158ff877-29c0-47c7-b769-6229e124701d admin admin] The token cache is being invalidate because project 55524dfedf924164a1d57f8b236c2afd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:21:09.942632 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-158ff877-29c0-47c7-b769-6229e124701d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:09.942632 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-158ff877-29c0-47c7-b769-6229e124701d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:09.943093 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 970/1924] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:21:09 2022] DELETE /identity/v3/projects/55524dfedf924164a1d57f8b236c2afd => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:09.944066 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 955/1925] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:09.944676 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48383d-d660-46d0-a225-029fa7166afe None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:09.945113 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48383d-d660-46d0-a225-029fa7166afe None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:09.945540 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48383d-d660-46d0-a225-029fa7166afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:09.946940 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e16ee5-7d20-49dd-b7f8-3ca6cb740c19 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:09.948337 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-05e16ee5-7d20-49dd-b7f8-3ca6cb740c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:09.961843 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e16ee5-7d20-49dd-b7f8-3ca6cb740c19 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:09.966755 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ecca7-6284-4969-90e3-59ea482205f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:09.967399 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ecca7-6284-4969-90e3-59ea482205f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:09.967725 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ecca7-6284-4969-90e3-59ea482205f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:09.968003 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ecca7-6284-4969-90e3-59ea482205f9 admin admin] PATH_INFO: `/v3/users/b832ab1f97d44060b5eb31765fd59c75` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:09.969185 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4ecca7-6284-4969-90e3-59ea482205f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b832ab1f97d44060b5eb31765fd59c75)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:09.971106 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4ecca7-6284-4969-90e3-59ea482205f9 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:09.971306 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-3a48383d-d660-46d0-a225-029fa7166afe None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:09.972722 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-3a48383d-d660-46d0-a225-029fa7166afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:09.980890 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 971/1926] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:09.982235 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a48383d-d660-46d0-a225-029fa7166afe None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:09.983650 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-3a48383d-d660-46d0-a225-029fa7166afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:10.039966 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a48383d-d660-46d0-a225-029fa7166afe None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:10.043524 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878cf399-703a-4374-b885-9a6e70fb7330 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:10.044124 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cf399-703a-4374-b885-9a6e70fb7330 service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:10.044928 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cf399-703a-4374-b885-9a6e70fb7330 service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:10.045196 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cf399-703a-4374-b885-9a6e70fb7330 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:10.046962 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-878cf399-703a-4374-b885-9a6e70fb7330 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:10.058519 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878cf399-703a-4374-b885-9a6e70fb7330 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:10.063115 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878cf399-703a-4374-b885-9a6e70fb7330 service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:10.070549 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 972/1927] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:21:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:21:10.071359 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878cf399-703a-4374-b885-9a6e70fb7330 service neutron] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:10.081186 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878cf399-703a-4374-b885-9a6e70fb7330 service neutron] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:10.085936 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ef9fa-27b1-44ba-b422-e699699d81ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:10.086538 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ef9fa-27b1-44ba-b422-e699699d81ef admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:10.086821 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ef9fa-27b1-44ba-b422-e699699d81ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:10.087143 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ef9fa-27b1-44ba-b422-e699699d81ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:10.087798 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ef9fa-27b1-44ba-b422-e699699d81ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:21:10.096081 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ef9fa-27b1-44ba-b422-e699699d81ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:10.096081 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ef9fa-27b1-44ba-b422-e699699d81ef admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:10.102232 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 973/1928] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:21:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:21:10.107502 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ef9fa-27b1-44ba-b422-e699699d81ef admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:10.118772 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ef9fa-27b1-44ba-b422-e699699d81ef admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:10.124620 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc42920-dcc3-4df3-b86f-38d9ceb2f874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:10.125230 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc42920-dcc3-4df3-b86f-38d9ceb2f874 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:10.125514 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc42920-dcc3-4df3-b86f-38d9ceb2f874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:10.125789 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc42920-dcc3-4df3-b86f-38d9ceb2f874 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:10.126589 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc42920-dcc3-4df3-b86f-38d9ceb2f874 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:10.128451 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc42920-dcc3-4df3-b86f-38d9ceb2f874 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:10.156294 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-6a4ecca7-6284-4969-90e3-59ea482205f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b832ab1f97d44060b5eb31765fd59c75'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:10.157062 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-6a4ecca7-6284-4969-90e3-59ea482205f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b832ab1f97d44060b5eb31765fd59c75'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:10.157715 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-6a4ecca7-6284-4969-90e3-59ea482205f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b832ab1f97d44060b5eb31765fd59c75'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:10.164890 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-6a4ecca7-6284-4969-90e3-59ea482205f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b832ab1f97d44060b5eb31765fd59c75'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:10.177825 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-6a4ecca7-6284-4969-90e3-59ea482205f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b832ab1f97d44060b5eb31765fd59c75'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:10.221802 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 974/1929] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:21:10 2022] POST /identity/v3/projects => generated 377 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:21:10.225868 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc42920-dcc3-4df3-b86f-38d9ceb2f874 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:10.227572 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-cbc42920-dcc3-4df3-b86f-38d9ceb2f874 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:10.247194 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 956/1930] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:21:09 2022] DELETE /identity/v3/users/b832ab1f97d44060b5eb31765fd59c75 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:21:10.312597 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc42920-dcc3-4df3-b86f-38d9ceb2f874 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:10.381812 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef7b77-5e9a-4f2e-9a84-2835a4b0349a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:10.382446 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef7b77-5e9a-4f2e-9a84-2835a4b0349a admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:10.382732 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef7b77-5e9a-4f2e-9a84-2835a4b0349a admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:10.383017 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef7b77-5e9a-4f2e-9a84-2835a4b0349a admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:10.383693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcef7b77-5e9a-4f2e-9a84-2835a4b0349a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:10.385881 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcef7b77-5e9a-4f2e-9a84-2835a4b0349a admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:10.496617 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 975/1931] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:21:10 2022] POST /identity/v3/users => generated 289 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:21:10.501079 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ecca7-6284-4969-90e3-59ea482205f9 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:10.516383 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ecca7-6284-4969-90e3-59ea482205f9 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:10.542658 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aea5ff-b34d-4efd-a035-53d8cae165d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:10.551845 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aea5ff-b34d-4efd-a035-53d8cae165d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:10.551845 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aea5ff-b34d-4efd-a035-53d8cae165d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:10.551845 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aea5ff-b34d-4efd-a035-53d8cae165d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:10.551845 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aea5ff-b34d-4efd-a035-53d8cae165d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:10.552663 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aea5ff-b34d-4efd-a035-53d8cae165d0 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:10.567748 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 957/1932] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:10 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:21:10.568734 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef7b77-5e9a-4f2e-9a84-2835a4b0349a admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:10.583826 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef7b77-5e9a-4f2e-9a84-2835a4b0349a admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:10.597369 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657fdaac-fbe4-4d58-9b88-a113eba0300d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:10.598519 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657fdaac-fbe4-4d58-9b88-a113eba0300d admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:10.598955 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657fdaac-fbe4-4d58-9b88-a113eba0300d admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:10.599392 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657fdaac-fbe4-4d58-9b88-a113eba0300d admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:10.600146 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657fdaac-fbe4-4d58-9b88-a113eba0300d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:10.603587 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657fdaac-fbe4-4d58-9b88-a113eba0300d admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:10.622092 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 976/1933] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:10 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:21:10.626503 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aea5ff-b34d-4efd-a035-53d8cae165d0 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:10.637811 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aea5ff-b34d-4efd-a035-53d8cae165d0 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:10.646429 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c4e1a-ad76-40fb-9c47-4882180d31a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:10.647440 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c4e1a-ad76-40fb-9c47-4882180d31a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:10.651819 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c4e1a-ad76-40fb-9c47-4882180d31a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:10.652577 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c4e1a-ad76-40fb-9c47-4882180d31a8 admin admin] PATH_INFO: `/v3/projects/5652962ee951443a8d432bb46fe1fd8a/users/5dce9587082a435093b849e4a712f2a3/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:10.673959 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795c4e1a-ad76-40fb-9c47-4882180d31a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5652962ee951443a8d432bb46fe1fd8a, user_id=5dce9587082a435093b849e4a712f2a3, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:10.679325 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795c4e1a-ad76-40fb-9c47-4882180d31a8 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:10.706234 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657fdaac-fbe4-4d58-9b88-a113eba0300d admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:10.708944 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-657fdaac-fbe4-4d58-9b88-a113eba0300d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:10.735607 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657fdaac-fbe4-4d58-9b88-a113eba0300d admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:10.742031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4e77c-32ba-47d5-881a-0bc566d24b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:10.743475 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4e77c-32ba-47d5-881a-0bc566d24b9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:10.743767 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4e77c-32ba-47d5-881a-0bc566d24b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:10.744103 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4e77c-32ba-47d5-881a-0bc566d24b9d admin admin] PATH_INFO: `/v3/projects/ed18129b40944cdda536f68868b860c1` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:10.746363 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac4e77c-32ba-47d5-881a-0bc566d24b9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed18129b40944cdda536f68868b860c1)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:10.749590 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac4e77c-32ba-47d5-881a-0bc566d24b9d admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:10.802480 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 958/1934] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:21:10 2022] PUT /identity/v3/projects/5652962ee951443a8d432bb46fe1fd8a/users/5dce9587082a435093b849e4a712f2a3/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:10.814485 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a7130-91ac-4d6a-a39a-b599afcd1f35 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:10.814887 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a7130-91ac-4d6a-a39a-b599afcd1f35 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:10.815240 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a7130-91ac-4d6a-a39a-b599afcd1f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:10.847004 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-0a6a7130-91ac-4d6a-a39a-b599afcd1f35 None None] MFA Rules not processed for user `5dce9587082a435093b849e4a712f2a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:10.848247 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-0a6a7130-91ac-4d6a-a39a-b599afcd1f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:10.905593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-7ac4e77c-32ba-47d5-881a-0bc566d24b9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed18129b40944cdda536f68868b860c1', 'request_id': 'req-7ac4e77c-32ba-47d5-881a-0bc566d24b9d'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:10.912016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-7ac4e77c-32ba-47d5-881a-0bc566d24b9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed18129b40944cdda536f68868b860c1', 'request_id': 'req-7ac4e77c-32ba-47d5-881a-0bc566d24b9d'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:10.936013 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 959/1935] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:10.940689 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a7130-91ac-4d6a-a39a-b599afcd1f35 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:10.996847 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-7ac4e77c-32ba-47d5-881a-0bc566d24b9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed18129b40944cdda536f68868b860c1', 'request_id': 'req-7ac4e77c-32ba-47d5-881a-0bc566d24b9d'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:11.011015 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a7130-91ac-4d6a-a39a-b599afcd1f35 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:11.011225 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-7ac4e77c-32ba-47d5-881a-0bc566d24b9d admin admin] The token cache is being invalidate because project ed18129b40944cdda536f68868b860c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:21:11.011629 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-7ac4e77c-32ba-47d5-881a-0bc566d24b9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:11.012461 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-7ac4e77c-32ba-47d5-881a-0bc566d24b9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:11.017614 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 977/1936] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:21:10 2022] DELETE /identity/v3/projects/ed18129b40944cdda536f68868b860c1 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:11.017614 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4e77c-32ba-47d5-881a-0bc566d24b9d admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:11.018253 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-7ac4e77c-32ba-47d5-881a-0bc566d24b9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:11.077737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6f7f2-e8e6-40b8-a61e-98a406405616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:11.078356 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6f7f2-e8e6-40b8-a61e-98a406405616 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:11.078645 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6f7f2-e8e6-40b8-a61e-98a406405616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:11.078925 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6f7f2-e8e6-40b8-a61e-98a406405616 admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:11.079614 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb6f7f2-e8e6-40b8-a61e-98a406405616 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:11.081591 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb6f7f2-e8e6-40b8-a61e-98a406405616 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:11.092206 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4e77c-32ba-47d5-881a-0bc566d24b9d admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:11.098096 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ee6f3-56f6-4cff-a5a3-e7e692552785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:11.098096 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ee6f3-56f6-4cff-a5a3-e7e692552785 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:11.098096 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ee6f3-56f6-4cff-a5a3-e7e692552785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:11.099166 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ee6f3-56f6-4cff-a5a3-e7e692552785 admin admin] PATH_INFO: `/v3/projects/939a72f1a4e74a36ba779d402750df84` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:11.100495 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ee6f3-56f6-4cff-a5a3-e7e692552785 admin admin] RBAC: Authorizing `identity:delete_project(project_id=939a72f1a4e74a36ba779d402750df84)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:11.102467 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ee6f3-56f6-4cff-a5a3-e7e692552785 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:11.216143 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 960/1937] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:21:10 2022] POST /identity/v3/projects => generated 375 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:21:11.217010 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6f7f2-e8e6-40b8-a61e-98a406405616 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:11.218597 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-3fb6f7f2-e8e6-40b8-a61e-98a406405616 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:11.262738 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-386ee6f3-56f6-4cff-a5a3-e7e692552785 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '939a72f1a4e74a36ba779d402750df84', 'request_id': 'req-386ee6f3-56f6-4cff-a5a3-e7e692552785'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:11.276630 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-386ee6f3-56f6-4cff-a5a3-e7e692552785 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '939a72f1a4e74a36ba779d402750df84', 'request_id': 'req-386ee6f3-56f6-4cff-a5a3-e7e692552785'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:11.284485 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6f7f2-e8e6-40b8-a61e-98a406405616 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:11.341099 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c4b5a-6e7d-490a-9598-9c0422c4b034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:11.341311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c4b5a-6e7d-490a-9598-9c0422c4b034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:11.346045 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c4b5a-6e7d-490a-9598-9c0422c4b034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:11.346045 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c4b5a-6e7d-490a-9598-9c0422c4b034 admin admin] PATH_INFO: `/v3/users/d033e6753a5e402083784ac82ab502b5` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:11.346045 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c4b5a-6e7d-490a-9598-9c0422c4b034 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d033e6753a5e402083784ac82ab502b5)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:11.346045 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c4b5a-6e7d-490a-9598-9c0422c4b034 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:11.365054 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-386ee6f3-56f6-4cff-a5a3-e7e692552785 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '939a72f1a4e74a36ba779d402750df84', 'request_id': 'req-386ee6f3-56f6-4cff-a5a3-e7e692552785'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:11.381157 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-386ee6f3-56f6-4cff-a5a3-e7e692552785 admin admin] The token cache is being invalidate because project 939a72f1a4e74a36ba779d402750df84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:21:11.381157 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-386ee6f3-56f6-4cff-a5a3-e7e692552785 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:11.381157 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-386ee6f3-56f6-4cff-a5a3-e7e692552785 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:11.381785 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 978/1938] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:21:11 2022] DELETE /identity/v3/projects/939a72f1a4e74a36ba779d402750df84 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:11.382533 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ee6f3-56f6-4cff-a5a3-e7e692552785 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:11.386290 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-386ee6f3-56f6-4cff-a5a3-e7e692552785 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:11.401841 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ee6f3-56f6-4cff-a5a3-e7e692552785 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:11.406778 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4720bc74-f8ad-4249-a3af-4e0c1e20f4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:11.407460 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720bc74-f8ad-4249-a3af-4e0c1e20f4c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:11.409129 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720bc74-f8ad-4249-a3af-4e0c1e20f4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:11.409129 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720bc74-f8ad-4249-a3af-4e0c1e20f4c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:11.409129 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4720bc74-f8ad-4249-a3af-4e0c1e20f4c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:11.410637 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4720bc74-f8ad-4249-a3af-4e0c1e20f4c6 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:11.503268 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 979/1939] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:21:11 2022] POST /identity/v3/users => generated 288 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:21:11.503268 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae68e6b1-c8fe-4ed6-b1a0-58229791f28d None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:11.503268 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae68e6b1-c8fe-4ed6-b1a0-58229791f28d None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:11.504584 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae68e6b1-c8fe-4ed6-b1a0-58229791f28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:11.520823 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-e62c4b5a-6e7d-490a-9598-9c0422c4b034 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd033e6753a5e402083784ac82ab502b5'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:11.521611 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-e62c4b5a-6e7d-490a-9598-9c0422c4b034 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd033e6753a5e402083784ac82ab502b5'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:11.522256 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-e62c4b5a-6e7d-490a-9598-9c0422c4b034 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd033e6753a5e402083784ac82ab502b5'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:11.527297 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-ae68e6b1-c8fe-4ed6-b1a0-58229791f28d None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:11.528554 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-ae68e6b1-c8fe-4ed6-b1a0-58229791f28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:11.529593 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-e62c4b5a-6e7d-490a-9598-9c0422c4b034 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd033e6753a5e402083784ac82ab502b5'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:11.544745 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 980/1940] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:11.545406 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-e62c4b5a-6e7d-490a-9598-9c0422c4b034 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd033e6753a5e402083784ac82ab502b5'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:11.549038 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae68e6b1-c8fe-4ed6-b1a0-58229791f28d None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:11.550574 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-ae68e6b1-c8fe-4ed6-b1a0-58229791f28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:11.563085 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae68e6b1-c8fe-4ed6-b1a0-58229791f28d None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:11.567823 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52450c34-c3c5-4bc8-b814-071299efafa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:11.568449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52450c34-c3c5-4bc8-b814-071299efafa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:11.568739 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52450c34-c3c5-4bc8-b814-071299efafa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:11.569018 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52450c34-c3c5-4bc8-b814-071299efafa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:11.569671 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52450c34-c3c5-4bc8-b814-071299efafa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:11.571271 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52450c34-c3c5-4bc8-b814-071299efafa0 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:11.582024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 981/1941] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:21:11.584304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d93e53-50b5-4e6d-8f40-74c59100f8f6 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:11.585553 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d93e53-50b5-4e6d-8f40-74c59100f8f6 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:11.585848 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d93e53-50b5-4e6d-8f40-74c59100f8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:11.606955 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-69d93e53-50b5-4e6d-8f40-74c59100f8f6 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:11.607933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-69d93e53-50b5-4e6d-8f40-74c59100f8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:11.616823 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 982/1942] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:11.617725 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d93e53-50b5-4e6d-8f40-74c59100f8f6 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:11.628268 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d93e53-50b5-4e6d-8f40-74c59100f8f6 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:11.633229 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15291269-be84-45b7-bda5-3afd7c4ad1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:11.633840 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15291269-be84-45b7-bda5-3afd7c4ad1a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:11.634121 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15291269-be84-45b7-bda5-3afd7c4ad1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:11.634399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15291269-be84-45b7-bda5-3afd7c4ad1a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:11.635332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15291269-be84-45b7-bda5-3afd7c4ad1a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:11.636742 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15291269-be84-45b7-bda5-3afd7c4ad1a9 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:11.651119 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 983/1943] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:21:11.651119 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15291269-be84-45b7-bda5-3afd7c4ad1a9 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:11.658820 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15291269-be84-45b7-bda5-3afd7c4ad1a9 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:11.663878 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6234b2e3-f6cc-4e1e-93c8-1fded1c34c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:11.664359 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234b2e3-f6cc-4e1e-93c8-1fded1c34c24 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:11.664639 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234b2e3-f6cc-4e1e-93c8-1fded1c34c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:11.664895 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234b2e3-f6cc-4e1e-93c8-1fded1c34c24 admin admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:11.665645 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6234b2e3-f6cc-4e1e-93c8-1fded1c34c24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:21:11.665997 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6234b2e3-f6cc-4e1e-93c8-1fded1c34c24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:11.667649 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6234b2e3-f6cc-4e1e-93c8-1fded1c34c24 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:11.674717 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 961/1944] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:21:11 2022] DELETE /identity/v3/users/d033e6753a5e402083784ac82ab502b5 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:11.675946 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c4b5a-6e7d-490a-9598-9c0422c4b034 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:11.681793 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 984/1945] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:21:11.686018 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6234b2e3-f6cc-4e1e-93c8-1fded1c34c24 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:11.708277 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6234b2e3-f6cc-4e1e-93c8-1fded1c34c24 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:11.756312 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:11.785173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:11.791646 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c4b5a-6e7d-490a-9598-9c0422c4b034 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:11.795542 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd66317-ba17-48df-959f-ab018c0a7ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:11.797473 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd66317-ba17-48df-959f-ab018c0a7ca4 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:11.797891 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd66317-ba17-48df-959f-ab018c0a7ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:11.798270 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd66317-ba17-48df-959f-ab018c0a7ca4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:11.798827 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b87276-a11e-4ecd-9cfe-324edeff045e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:11.799115 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd66317-ba17-48df-959f-ab018c0a7ca4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:11.799939 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b87276-a11e-4ecd-9cfe-324edeff045e admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:11.800224 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b87276-a11e-4ecd-9cfe-324edeff045e admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:11.800506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b87276-a11e-4ecd-9cfe-324edeff045e admin admin] PATH_INFO: `/v3/projects/973cabad6bba47fd9628b80ef05771eb/users/40984d4b52154ccf912552adf98ecdb3/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:11.801584 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd66317-ba17-48df-959f-ab018c0a7ca4 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:11.816644 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b87276-a11e-4ecd-9cfe-324edeff045e admin admin] RBAC: Authorizing `identity:create_grant(project_id=973cabad6bba47fd9628b80ef05771eb, user_id=40984d4b52154ccf912552adf98ecdb3, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:11.819485 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b87276-a11e-4ecd-9cfe-324edeff045e admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:11.916787 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 962/1946] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:21:11 2022] PUT /identity/v3/projects/973cabad6bba47fd9628b80ef05771eb/users/40984d4b52154ccf912552adf98ecdb3/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:11.918177 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 985/1947] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:21:11 2022] POST /identity/v3/projects => generated 385 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:21:11.922355 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b87276-a11e-4ecd-9cfe-324edeff045e admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:11.927494 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e36e19c-7cc1-4a45-82a5-d88aaf7752bf None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:11.927790 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e36e19c-7cc1-4a45-82a5-d88aaf7752bf None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:11.928077 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e36e19c-7cc1-4a45-82a5-d88aaf7752bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:11.985352 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-8e36e19c-7cc1-4a45-82a5-d88aaf7752bf None None] MFA Rules not processed for user `40984d4b52154ccf912552adf98ecdb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:11.986730 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-8e36e19c-7cc1-4a45-82a5-d88aaf7752bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:12.003660 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b87276-a11e-4ecd-9cfe-324edeff045e admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:12.075387 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae9f6a-ef06-43ee-95ad-f48ac8d11073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:12.075713 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 986/1948] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:21:12.076132 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae9f6a-ef06-43ee-95ad-f48ac8d11073 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:12.076334 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae9f6a-ef06-43ee-95ad-f48ac8d11073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:12.076711 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae9f6a-ef06-43ee-95ad-f48ac8d11073 admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:12.077838 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae9f6a-ef06-43ee-95ad-f48ac8d11073 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:12.080018 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae9f6a-ef06-43ee-95ad-f48ac8d11073 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:12.087370 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e36e19c-7cc1-4a45-82a5-d88aaf7752bf None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:12.103053 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e36e19c-7cc1-4a45-82a5-d88aaf7752bf None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:12.107836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc09b85-d901-488e-901b-6d5248856971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:12.109698 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc09b85-d901-488e-901b-6d5248856971 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:12.110135 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc09b85-d901-488e-901b-6d5248856971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:12.110401 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc09b85-d901-488e-901b-6d5248856971 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:12.111039 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc09b85-d901-488e-901b-6d5248856971 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:12.113062 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc09b85-d901-488e-901b-6d5248856971 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:12.185963 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 963/1949] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:21:11 2022] POST /identity/v3/users => generated 293 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:21:12.188356 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae9f6a-ef06-43ee-95ad-f48ac8d11073 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:12.199952 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae9f6a-ef06-43ee-95ad-f48ac8d11073 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:12.205429 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a574ed3-a04d-4a11-bb57-06315d69f43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:12.206044 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a574ed3-a04d-4a11-bb57-06315d69f43d admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:12.206365 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a574ed3-a04d-4a11-bb57-06315d69f43d admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:12.206674 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a574ed3-a04d-4a11-bb57-06315d69f43d admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:12.212564 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a574ed3-a04d-4a11-bb57-06315d69f43d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:12.212564 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a574ed3-a04d-4a11-bb57-06315d69f43d admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:12.219881 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 964/1950] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:21:12.224282 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a574ed3-a04d-4a11-bb57-06315d69f43d admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:12.235710 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a574ed3-a04d-4a11-bb57-06315d69f43d admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:12.240752 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e04eb5-e228-467a-92a6-b66e8cf32c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:12.241376 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e04eb5-e228-467a-92a6-b66e8cf32c58 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:12.241662 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e04eb5-e228-467a-92a6-b66e8cf32c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:12.241940 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e04eb5-e228-467a-92a6-b66e8cf32c58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:12.242587 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e04eb5-e228-467a-92a6-b66e8cf32c58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:12.244218 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e04eb5-e228-467a-92a6-b66e8cf32c58 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:12.246222 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 987/1951] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:21:12 2022] POST /identity/v3/projects => generated 375 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:21:12.250144 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc09b85-d901-488e-901b-6d5248856971 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:12.259040 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 965/1952] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 12:21:12.260395 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e04eb5-e228-467a-92a6-b66e8cf32c58 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:12.272239 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e04eb5-e228-467a-92a6-b66e8cf32c58 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:12.318975 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc09b85-d901-488e-901b-6d5248856971 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:12.326507 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e49fd-126f-4afa-a311-d4fc0c39ac13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:12.327936 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e49fd-126f-4afa-a311-d4fc0c39ac13 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:12.328409 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e49fd-126f-4afa-a311-d4fc0c39ac13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:12.328792 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e49fd-126f-4afa-a311-d4fc0c39ac13 admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:12.329474 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e49fd-126f-4afa-a311-d4fc0c39ac13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:12.338719 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e49fd-126f-4afa-a311-d4fc0c39ac13 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:12.342311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23049a4-ef28-43a8-998f-e568cd57ac13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:12.343991 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23049a4-ef28-43a8-998f-e568cd57ac13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:12.344433 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23049a4-ef28-43a8-998f-e568cd57ac13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:12.345283 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23049a4-ef28-43a8-998f-e568cd57ac13 admin admin] PATH_INFO: `/v3/projects/3df4b5b5f690447b8638e036bb602f99/users/74c492edc07f4dc99f0a4d6d7fbe54ba/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:12.374462 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23049a4-ef28-43a8-998f-e568cd57ac13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3df4b5b5f690447b8638e036bb602f99, user_id=74c492edc07f4dc99f0a4d6d7fbe54ba, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:12.377615 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23049a4-ef28-43a8-998f-e568cd57ac13 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:12.444066 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 988/1953] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:21:12 2022] POST /identity/v3/users => generated 294 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:21:12.444314 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e49fd-126f-4afa-a311-d4fc0c39ac13 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:12.445879 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-883e49fd-126f-4afa-a311-d4fc0c39ac13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:12.458536 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e49fd-126f-4afa-a311-d4fc0c39ac13 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:12.463201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de6d48d-25ef-4007-8f84-337432250412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:12.463886 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6d48d-25ef-4007-8f84-337432250412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:12.469801 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6d48d-25ef-4007-8f84-337432250412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:12.469801 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6d48d-25ef-4007-8f84-337432250412 admin admin] PATH_INFO: `/v3/projects/cf8bd977d56a457a85b3993e4ca0663d` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:12.469801 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de6d48d-25ef-4007-8f84-337432250412 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf8bd977d56a457a85b3993e4ca0663d)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:12.469801 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de6d48d-25ef-4007-8f84-337432250412 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:12.472448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 966/1954] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:21:12 2022] PUT /identity/v3/projects/3df4b5b5f690447b8638e036bb602f99/users/74c492edc07f4dc99f0a4d6d7fbe54ba/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:12.473423 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23049a4-ef28-43a8-998f-e568cd57ac13 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:12.542753 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23049a4-ef28-43a8-998f-e568cd57ac13 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:12.548116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb233e7-1795-4a34-bf1a-1fb71e8fe184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:12.548783 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb233e7-1795-4a34-bf1a-1fb71e8fe184 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:12.549082 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb233e7-1795-4a34-bf1a-1fb71e8fe184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:12.549350 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb233e7-1795-4a34-bf1a-1fb71e8fe184 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:12.553773 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb233e7-1795-4a34-bf1a-1fb71e8fe184 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:12.553773 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb233e7-1795-4a34-bf1a-1fb71e8fe184 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:12.562825 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 967/1955] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:21:12.567481 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5efd065-45f8-45eb-b080-f49af98b6c54 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:12.567481 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5efd065-45f8-45eb-b080-f49af98b6c54 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:12.567481 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5efd065-45f8-45eb-b080-f49af98b6c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:12.598422 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-a5efd065-45f8-45eb-b080-f49af98b6c54 None None] MFA Rules not processed for user `74c492edc07f4dc99f0a4d6d7fbe54ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:12.599710 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-a5efd065-45f8-45eb-b080-f49af98b6c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:12.611380 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-7de6d48d-25ef-4007-8f84-337432250412 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf8bd977d56a457a85b3993e4ca0663d', 'request_id': 'req-7de6d48d-25ef-4007-8f84-337432250412'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:12.621622 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-7de6d48d-25ef-4007-8f84-337432250412 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf8bd977d56a457a85b3993e4ca0663d', 'request_id': 'req-7de6d48d-25ef-4007-8f84-337432250412'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:12.696613 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 968/1956] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:12.697547 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5efd065-45f8-45eb-b080-f49af98b6c54 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:12.735644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-7de6d48d-25ef-4007-8f84-337432250412 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf8bd977d56a457a85b3993e4ca0663d', 'request_id': 'req-7de6d48d-25ef-4007-8f84-337432250412'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:12.770356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-7de6d48d-25ef-4007-8f84-337432250412 admin admin] The token cache is being invalidate because project cf8bd977d56a457a85b3993e4ca0663d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:21:12.772606 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-7de6d48d-25ef-4007-8f84-337432250412 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:12.772606 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-7de6d48d-25ef-4007-8f84-337432250412 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:12.773456 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 989/1957] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:21:12 2022] DELETE /identity/v3/projects/cf8bd977d56a457a85b3993e4ca0663d => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:12.774224 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de6d48d-25ef-4007-8f84-337432250412 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:12.776330 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5efd065-45f8-45eb-b080-f49af98b6c54 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:12.779668 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-7de6d48d-25ef-4007-8f84-337432250412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:12.786830 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b913dca-afa1-4dd8-97eb-c704c1ca41c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:12.787626 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b913dca-afa1-4dd8-97eb-c704c1ca41c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:12.788134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b913dca-afa1-4dd8-97eb-c704c1ca41c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:12.788486 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b913dca-afa1-4dd8-97eb-c704c1ca41c1 admin admin] PATH_INFO: `/v3/projects/f7cf769018dc4d268ca94cdc49c9c037/users/6b620f862ebb4e62bf90fd0ccb9e281c/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:12.793988 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de6d48d-25ef-4007-8f84-337432250412 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:12.799265 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43dcbb-4381-40ea-af63-95afb70b2c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:12.799938 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43dcbb-4381-40ea-af63-95afb70b2c6f admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:12.800222 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43dcbb-4381-40ea-af63-95afb70b2c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:12.801299 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43dcbb-4381-40ea-af63-95afb70b2c6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:12.801299 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a43dcbb-4381-40ea-af63-95afb70b2c6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:12.803144 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a43dcbb-4381-40ea-af63-95afb70b2c6f admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:12.807597 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b913dca-afa1-4dd8-97eb-c704c1ca41c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7cf769018dc4d268ca94cdc49c9c037, user_id=6b620f862ebb4e62bf90fd0ccb9e281c, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:12.810266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b913dca-afa1-4dd8-97eb-c704c1ca41c1 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:12.821400 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:12.896452 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 969/1958] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:21:12 2022] PUT /identity/v3/projects/f7cf769018dc4d268ca94cdc49c9c037/users/6b620f862ebb4e62bf90fd0ccb9e281c/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:12.897867 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7d19c-535b-420f-bcea-3fd9f9e30a48 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:12.898234 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7d19c-535b-420f-bcea-3fd9f9e30a48 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:12.904543 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7d19c-535b-420f-bcea-3fd9f9e30a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:12.922279 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-b9a7d19c-535b-420f-bcea-3fd9f9e30a48 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:12.923969 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-b9a7d19c-535b-420f-bcea-3fd9f9e30a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:12.940515 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 990/1959] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:21:12 2022] POST /identity/v3/projects => generated 385 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:21:12.942174 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43dcbb-4381-40ea-af63-95afb70b2c6f admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:12.945348 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-3a43dcbb-4381-40ea-af63-95afb70b2c6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:13.023559 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43dcbb-4381-40ea-af63-95afb70b2c6f admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:13.030206 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17705912-1f95-443b-ac14-290eccd82f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:13.030838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17705912-1f95-443b-ac14-290eccd82f89 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:13.032832 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17705912-1f95-443b-ac14-290eccd82f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:13.032832 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17705912-1f95-443b-ac14-290eccd82f89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:13.032832 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17705912-1f95-443b-ac14-290eccd82f89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:13.033796 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17705912-1f95-443b-ac14-290eccd82f89 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:13.042717 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 970/1960] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:13.043756 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7d19c-535b-420f-bcea-3fd9f9e30a48 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:13.054050 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 991/1961] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:21:13.056000 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ccb36c-0cc2-427a-bb0e-73d4f327eb8e None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:13.056301 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ccb36c-0cc2-427a-bb0e-73d4f327eb8e None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:13.056586 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ccb36c-0cc2-427a-bb0e-73d4f327eb8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:13.062984 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7d19c-535b-420f-bcea-3fd9f9e30a48 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:13.069490 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005cf69a-e0eb-471f-9ae8-f09036026031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:13.070070 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005cf69a-e0eb-471f-9ae8-f09036026031 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:13.070355 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005cf69a-e0eb-471f-9ae8-f09036026031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:13.070635 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005cf69a-e0eb-471f-9ae8-f09036026031 admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:13.071277 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005cf69a-e0eb-471f-9ae8-f09036026031 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:13.073553 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005cf69a-e0eb-471f-9ae8-f09036026031 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:13.086966 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-17ccb36c-0cc2-427a-bb0e-73d4f327eb8e None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:13.088347 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-17ccb36c-0cc2-427a-bb0e-73d4f327eb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:13.099713 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 992/1962] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:13.103884 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ccb36c-0cc2-427a-bb0e-73d4f327eb8e None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:13.110648 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:INSERT=4,keystone:SELECT=81,keystone:DELETE=15 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:13.114190 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ccb36c-0cc2-427a-bb0e-73d4f327eb8e None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:13.118941 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2ecd8-1318-4890-bb06-b0d505fd7225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:13.119836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2ecd8-1318-4890-bb06-b0d505fd7225 admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:13.120279 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2ecd8-1318-4890-bb06-b0d505fd7225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:13.120561 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2ecd8-1318-4890-bb06-b0d505fd7225 admin admin] PATH_INFO: `/v3/projects/f7cf769018dc4d268ca94cdc49c9c037/users/6b620f862ebb4e62bf90fd0ccb9e281c/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:13.122771 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2ecd8-1318-4890-bb06-b0d505fd7225 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7cf769018dc4d268ca94cdc49c9c037, user_id=6b620f862ebb4e62bf90fd0ccb9e281c, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:13.126072 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2ecd8-1318-4890-bb06-b0d505fd7225 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:13.139272 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 993/1963] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:21:13 2022] PUT /identity/v3/projects/f7cf769018dc4d268ca94cdc49c9c037/users/6b620f862ebb4e62bf90fd0ccb9e281c/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:13.139272 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2ecd8-1318-4890-bb06-b0d505fd7225 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:13.180373 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 971/1964] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:21:13 2022] POST /identity/v3/users => generated 299 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:21:13.181881 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f9d60a-63c0-4dee-b18d-d1b398c99fb1 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:13.182175 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f9d60a-63c0-4dee-b18d-d1b398c99fb1 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:13.182460 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f9d60a-63c0-4dee-b18d-d1b398c99fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:13.190383 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:13.227344 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-37f9d60a-63c0-4dee-b18d-d1b398c99fb1 None None] MFA Rules not processed for user `6b620f862ebb4e62bf90fd0ccb9e281c`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:13.228637 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-37f9d60a-63c0-4dee-b18d-d1b398c99fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:13.231115 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2ecd8-1318-4890-bb06-b0d505fd7225 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:13.236349 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77282352-2950-43c1-b755-4042c3cfd0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:13.236961 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77282352-2950-43c1-b755-4042c3cfd0d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:13.237243 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77282352-2950-43c1-b755-4042c3cfd0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:13.237513 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77282352-2950-43c1-b755-4042c3cfd0d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:13.238291 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77282352-2950-43c1-b755-4042c3cfd0d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:21:13.238597 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77282352-2950-43c1-b755-4042c3cfd0d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:13.240230 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77282352-2950-43c1-b755-4042c3cfd0d7 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:13.252881 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 994/1965] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:21:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:21:13.253874 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77282352-2950-43c1-b755-4042c3cfd0d7 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:13.265041 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77282352-2950-43c1-b755-4042c3cfd0d7 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:13.270098 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b2bf2e-3839-47cc-b2a9-c10587bc66b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:13.273004 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2bf2e-3839-47cc-b2a9-c10587bc66b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:13.273384 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2bf2e-3839-47cc-b2a9-c10587bc66b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:13.273667 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2bf2e-3839-47cc-b2a9-c10587bc66b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:13.274562 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b2bf2e-3839-47cc-b2a9-c10587bc66b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:13.276329 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b2bf2e-3839-47cc-b2a9-c10587bc66b8 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:13.293132 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 995/1966] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:21:13.293132 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b2bf2e-3839-47cc-b2a9-c10587bc66b8 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:13.305453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b2bf2e-3839-47cc-b2a9-c10587bc66b8 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:13.311500 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 972/1967] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:13 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:13.312376 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f4dc7c-fc85-4e87-95c3-cdb5189690cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:13.312509 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f9d60a-63c0-4dee-b18d-d1b398c99fb1 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:13.312993 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4dc7c-fc85-4e87-95c3-cdb5189690cf admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:13.313289 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4dc7c-fc85-4e87-95c3-cdb5189690cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:13.313565 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4dc7c-fc85-4e87-95c3-cdb5189690cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:13.314296 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f4dc7c-fc85-4e87-95c3-cdb5189690cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:13.316330 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f4dc7c-fc85-4e87-95c3-cdb5189690cf admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:13.323662 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f9d60a-63c0-4dee-b18d-d1b398c99fb1 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:13.328931 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d33bfa-435a-486e-bb4f-7bd994e73217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:13.329644 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d33bfa-435a-486e-bb4f-7bd994e73217 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:13.329926 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d33bfa-435a-486e-bb4f-7bd994e73217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:13.330497 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d33bfa-435a-486e-bb4f-7bd994e73217 admin admin] PATH_INFO: `/v3/projects/cba4a46eeef84dc4940b41d2a93e2a3d/users/49427c7a7ba74cea9f4d4c76635ae98b/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:13.347509 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d33bfa-435a-486e-bb4f-7bd994e73217 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cba4a46eeef84dc4940b41d2a93e2a3d, user_id=49427c7a7ba74cea9f4d4c76635ae98b, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:13.350208 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d33bfa-435a-486e-bb4f-7bd994e73217 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:13.423157 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 996/1968] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:21:13 2022] POST /identity/v3/projects => generated 365 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:21:13.424224 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f4dc7c-fc85-4e87-95c3-cdb5189690cf admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:13.427173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-75f4dc7c-fc85-4e87-95c3-cdb5189690cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:13.449114 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 973/1969] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:21:13 2022] PUT /identity/v3/projects/cba4a46eeef84dc4940b41d2a93e2a3d/users/49427c7a7ba74cea9f4d4c76635ae98b/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:13.449945 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d33bfa-435a-486e-bb4f-7bd994e73217 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:13.489894 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f4dc7c-fc85-4e87-95c3-cdb5189690cf admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:13.518332 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d33bfa-435a-486e-bb4f-7bd994e73217 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:13.523623 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2a826-5287-49d6-aee6-0793f53589ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:13.524314 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2a826-5287-49d6-aee6-0793f53589ed admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:13.524600 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2a826-5287-49d6-aee6-0793f53589ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:13.524879 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2a826-5287-49d6-aee6-0793f53589ed admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:13.525512 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb2a826-5287-49d6-aee6-0793f53589ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:13.527645 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb2a826-5287-49d6-aee6-0793f53589ed admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:13.540457 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f51264b-c85e-4ff2-9458-54559dab5b7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:13.541085 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51264b-c85e-4ff2-9458-54559dab5b7f service cinder] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:13.541371 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51264b-c85e-4ff2-9458-54559dab5b7f service cinder] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:13.541649 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51264b-c85e-4ff2-9458-54559dab5b7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:13.543482 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-1f51264b-c85e-4ff2-9458-54559dab5b7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:13.616883 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f51264b-c85e-4ff2-9458-54559dab5b7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:13.618327 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f51264b-c85e-4ff2-9458-54559dab5b7f service cinder] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:13.628910 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 997/1970] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:21:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:21:13.629904 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f51264b-c85e-4ff2-9458-54559dab5b7f service cinder] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:13.643368 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f51264b-c85e-4ff2-9458-54559dab5b7f service cinder] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:13.649574 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6f864-f361-4b8e-aa71-afa8c1502703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:13.650211 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6f864-f361-4b8e-aa71-afa8c1502703 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:13.650532 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6f864-f361-4b8e-aa71-afa8c1502703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:13.650813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6f864-f361-4b8e-aa71-afa8c1502703 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:13.651492 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a6f864-f361-4b8e-aa71-afa8c1502703 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:13.653102 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a6f864-f361-4b8e-aa71-afa8c1502703 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:13.664427 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 998/1971] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:21:13.669406 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6f864-f361-4b8e-aa71-afa8c1502703 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:13.680329 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6f864-f361-4b8e-aa71-afa8c1502703 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:13.685695 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4937a1f7-ce7f-477e-a8fc-3297da8bfbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:13.687132 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937a1f7-ce7f-477e-a8fc-3297da8bfbd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:13.687480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937a1f7-ce7f-477e-a8fc-3297da8bfbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:13.687771 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937a1f7-ce7f-477e-a8fc-3297da8bfbd6 admin admin] PATH_INFO: `/v3/projects/cba4a46eeef84dc4940b41d2a93e2a3d/users/49427c7a7ba74cea9f4d4c76635ae98b/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:13.696122 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4937a1f7-ce7f-477e-a8fc-3297da8bfbd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cba4a46eeef84dc4940b41d2a93e2a3d, user_id=49427c7a7ba74cea9f4d4c76635ae98b, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:13.698970 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4937a1f7-ce7f-477e-a8fc-3297da8bfbd6 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:13.712836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 999/1972] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:21:13 2022] PUT /identity/v3/projects/cba4a46eeef84dc4940b41d2a93e2a3d/users/49427c7a7ba74cea9f4d4c76635ae98b/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:21:13.728869 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcae25c-01cb-4070-847e-144b1df8a6d7 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:13.732677 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcae25c-01cb-4070-847e-144b1df8a6d7 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:13.732677 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcae25c-01cb-4070-847e-144b1df8a6d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:13.756772 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 974/1973] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:21:13 2022] POST /identity/v3/users => generated 283 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:21:13.759305 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2a826-5287-49d6-aee6-0793f53589ed admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:13.766582 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-5dcae25c-01cb-4070-847e-144b1df8a6d7 None None] MFA Rules not processed for user `49427c7a7ba74cea9f4d4c76635ae98b`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:13.767979 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-5dcae25c-01cb-4070-847e-144b1df8a6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:13.784933 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2a826-5287-49d6-aee6-0793f53589ed admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:13.878122 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:13.880776 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1000/1974] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:13 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:21:13.899562 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcae25c-01cb-4070-847e-144b1df8a6d7 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:13.901508 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-5dcae25c-01cb-4070-847e-144b1df8a6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:13.914275 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa8a30-3ae9-4da4-afc8-d84f603d6c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:13.915148 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa8a30-3ae9-4da4-afc8-d84f603d6c7e admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:13.915616 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa8a30-3ae9-4da4-afc8-d84f603d6c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:13.915947 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa8a30-3ae9-4da4-afc8-d84f603d6c7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:13.916732 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa8a30-3ae9-4da4-afc8-d84f603d6c7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:13.918429 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa8a30-3ae9-4da4-afc8-d84f603d6c7e admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:13.937614 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 975/1975] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:21:13.942821 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa8a30-3ae9-4da4-afc8-d84f603d6c7e admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:13.960028 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa8a30-3ae9-4da4-afc8-d84f603d6c7e admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:13.974410 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3b0ec-87de-47ce-bf6a-b00382e2d3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:13.975764 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3b0ec-87de-47ce-bf6a-b00382e2d3d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:13.976073 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3b0ec-87de-47ce-bf6a-b00382e2d3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:13.976360 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3b0ec-87de-47ce-bf6a-b00382e2d3d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:13.977475 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb3b0ec-87de-47ce-bf6a-b00382e2d3d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:13.981016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb3b0ec-87de-47ce-bf6a-b00382e2d3d2 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:13.992986 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:14.003106 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcae25c-01cb-4070-847e-144b1df8a6d7 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:14.003699 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c719885-830b-4068-a938-ef142abdd6fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:14.004283 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 976/1976] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 12:21:14.004417 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c719885-830b-4068-a938-ef142abdd6fb service cinder] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:14.004707 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c719885-830b-4068-a938-ef142abdd6fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:14.004994 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c719885-830b-4068-a938-ef142abdd6fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:14.009104 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3b0ec-87de-47ce-bf6a-b00382e2d3d2 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:14.021958 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c719885-830b-4068-a938-ef142abdd6fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:14.024669 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c719885-830b-4068-a938-ef142abdd6fb service cinder] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:14.041439 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3b0ec-87de-47ce-bf6a-b00382e2d3d2 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:14.046063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1001/1977] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:21:13 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:21:14.050383 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f680c-9104-4943-bca2-381c21bfbe84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:14.051552 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f680c-9104-4943-bca2-381c21bfbe84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:14.052434 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f680c-9104-4943-bca2-381c21bfbe84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:14.052913 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f680c-9104-4943-bca2-381c21bfbe84 admin admin] PATH_INFO: `/v3/projects/76083b06f8174610b8adae0609240077/users/d756e53d759a4535b580cb84a429520d/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:14.073831 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f680c-9104-4943-bca2-381c21bfbe84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76083b06f8174610b8adae0609240077, user_id=d756e53d759a4535b580cb84a429520d, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:14.077611 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f680c-9104-4943-bca2-381c21bfbe84 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:14.090840 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c719885-830b-4068-a938-ef142abdd6fb service cinder] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:14.104561 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c719885-830b-4068-a938-ef142abdd6fb service cinder] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:14.112273 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bedc06-abe7-4031-918c-7502bdde0745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:14.115834 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bedc06-abe7-4031-918c-7502bdde0745 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:14.115834 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bedc06-abe7-4031-918c-7502bdde0745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:14.115834 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bedc06-abe7-4031-918c-7502bdde0745 admin admin] PATH_INFO: `/v3/users/74c492edc07f4dc99f0a4d6d7fbe54ba` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:14.115834 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bedc06-abe7-4031-918c-7502bdde0745 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74c492edc07f4dc99f0a4d6d7fbe54ba)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:14.117764 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bedc06-abe7-4031-918c-7502bdde0745 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:14.177955 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 977/1978] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:21:14 2022] PUT /identity/v3/projects/76083b06f8174610b8adae0609240077/users/d756e53d759a4535b580cb84a429520d/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:14.190471 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbbaf2-6890-460c-9890-0dea4aa86f40 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:14.190804 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbbaf2-6890-460c-9890-0dea4aa86f40 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:14.191187 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbbaf2-6890-460c-9890-0dea4aa86f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:14.258129 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-c6dbbaf2-6890-460c-9890-0dea4aa86f40 None None] MFA Rules not processed for user `d756e53d759a4535b580cb84a429520d`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:14.259644 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-c6dbbaf2-6890-460c-9890-0dea4aa86f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:14.347018 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-73bedc06-abe7-4031-918c-7502bdde0745 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74c492edc07f4dc99f0a4d6d7fbe54ba'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:14.362838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-73bedc06-abe7-4031-918c-7502bdde0745 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74c492edc07f4dc99f0a4d6d7fbe54ba'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:14.364732 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-73bedc06-abe7-4031-918c-7502bdde0745 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74c492edc07f4dc99f0a4d6d7fbe54ba'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:14.371153 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 978/1979] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:21:14.376003 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dbbaf2-6890-460c-9890-0dea4aa86f40 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:14.387290 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-73bedc06-abe7-4031-918c-7502bdde0745 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74c492edc07f4dc99f0a4d6d7fbe54ba'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:14.462260 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dbbaf2-6890-460c-9890-0dea4aa86f40 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:14.467235 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc739dcf-b5ec-4e8a-944e-f7315519d03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:14.468554 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc739dcf-b5ec-4e8a-944e-f7315519d03a admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:14.469055 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc739dcf-b5ec-4e8a-944e-f7315519d03a admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:14.469336 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc739dcf-b5ec-4e8a-944e-f7315519d03a admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:14.470022 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc739dcf-b5ec-4e8a-944e-f7315519d03a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:14.470641 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-73bedc06-abe7-4031-918c-7502bdde0745 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74c492edc07f4dc99f0a4d6d7fbe54ba'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:14.472108 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc739dcf-b5ec-4e8a-944e-f7315519d03a admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:14.478722 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1002/1980] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:21:14 2022] DELETE /identity/v3/users/74c492edc07f4dc99f0a4d6d7fbe54ba => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:14.482935 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429deaa-8e12-4fa9-b593-c667c0dae4c5 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:14.483495 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429deaa-8e12-4fa9-b593-c667c0dae4c5 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:14.484228 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429deaa-8e12-4fa9-b593-c667c0dae4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:14.506594 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-4429deaa-8e12-4fa9-b593-c667c0dae4c5 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:14.508233 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-4429deaa-8e12-4fa9-b593-c667c0dae4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:14.567606 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 979/1981] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:21:14 2022] POST /identity/v3/projects => generated 365 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:21:14.568759 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc739dcf-b5ec-4e8a-944e-f7315519d03a admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:14.570476 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-fc739dcf-b5ec-4e8a-944e-f7315519d03a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:14.573704 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1003/1982] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:21:14.580926 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4429deaa-8e12-4fa9-b593-c667c0dae4c5 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:14.582339 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-4429deaa-8e12-4fa9-b593-c667c0dae4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:14.639871 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc739dcf-b5ec-4e8a-944e-f7315519d03a admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:14.640257 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4429deaa-8e12-4fa9-b593-c667c0dae4c5 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:14.644698 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ac051e-99f2-4102-8470-7c4ec3fb0ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:14.645312 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac051e-99f2-4102-8470-7c4ec3fb0ff7 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:14.645597 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac051e-99f2-4102-8470-7c4ec3fb0ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:14.645861 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac051e-99f2-4102-8470-7c4ec3fb0ff7 admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:14.646629 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ac051e-99f2-4102-8470-7c4ec3fb0ff7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:14.648609 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ac051e-99f2-4102-8470-7c4ec3fb0ff7 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:14.695230 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92fabba-d846-48ca-a367-e898d145a86e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:14.695778 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fabba-d846-48ca-a367-e898d145a86e service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:14.696061 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fabba-d846-48ca-a367-e898d145a86e service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:14.696338 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fabba-d846-48ca-a367-e898d145a86e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:14.707351 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92fabba-d846-48ca-a367-e898d145a86e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:14.708958 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92fabba-d846-48ca-a367-e898d145a86e service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:14.720730 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1004/1983] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:21:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:21:14.776205 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 980/1984] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:21:14 2022] POST /identity/v3/users => generated 289 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:21:14.781072 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92fabba-d846-48ca-a367-e898d145a86e service neutron] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:14.791622 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92fabba-d846-48ca-a367-e898d145a86e service neutron] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:14.796839 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291762d-c929-4b4a-ba4a-5616e25af287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:14.797481 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291762d-c929-4b4a-ba4a-5616e25af287 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:14.797767 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291762d-c929-4b4a-ba4a-5616e25af287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:14.798044 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291762d-c929-4b4a-ba4a-5616e25af287 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:14.798699 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2291762d-c929-4b4a-ba4a-5616e25af287 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:14.800316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2291762d-c929-4b4a-ba4a-5616e25af287 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:14.810832 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1005/1985] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:21:14.816188 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ac051e-99f2-4102-8470-7c4ec3fb0ff7 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:14.828158 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ac051e-99f2-4102-8470-7c4ec3fb0ff7 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:14.833473 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44910898-46ae-4d3e-82c7-20f5e1e77633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:14.834387 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44910898-46ae-4d3e-82c7-20f5e1e77633 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:14.835078 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44910898-46ae-4d3e-82c7-20f5e1e77633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:14.835192 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44910898-46ae-4d3e-82c7-20f5e1e77633 admin admin] PATH_INFO: `/v3/projects/ca95f9666c62498dbaaea4aeafb4a152/users/c32582762c9946e29f32c7ff35260aa5/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:14.865883 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44910898-46ae-4d3e-82c7-20f5e1e77633 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca95f9666c62498dbaaea4aeafb4a152, user_id=c32582762c9946e29f32c7ff35260aa5, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:14.868639 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44910898-46ae-4d3e-82c7-20f5e1e77633 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:14.962881 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 981/1986] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:21:14 2022] PUT /identity/v3/projects/ca95f9666c62498dbaaea4aeafb4a152/users/c32582762c9946e29f32c7ff35260aa5/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:14.964219 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291762d-c929-4b4a-ba4a-5616e25af287 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:14.976525 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291762d-c929-4b4a-ba4a-5616e25af287 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:15.018856 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:15.038752 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19f864-0822-4c13-800b-268992e9d87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:15.039392 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19f864-0822-4c13-800b-268992e9d87e admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:15.039712 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19f864-0822-4c13-800b-268992e9d87e admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:15.040019 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19f864-0822-4c13-800b-268992e9d87e admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:15.040702 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19f864-0822-4c13-800b-268992e9d87e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:15.042319 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19f864-0822-4c13-800b-268992e9d87e admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:15.052795 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1006/1987] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:14 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 12:21:15.058063 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44910898-46ae-4d3e-82c7-20f5e1e77633 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:15.071456 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44910898-46ae-4d3e-82c7-20f5e1e77633 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:15.075957 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03312cf3-334c-4880-93a9-46b26fc5a65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:15.076635 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03312cf3-334c-4880-93a9-46b26fc5a65c admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:15.077546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03312cf3-334c-4880-93a9-46b26fc5a65c admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:15.078356 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03312cf3-334c-4880-93a9-46b26fc5a65c admin admin] PATH_INFO: `/v3/projects/ca95f9666c62498dbaaea4aeafb4a152/users/c32582762c9946e29f32c7ff35260aa5/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:15.080014 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03312cf3-334c-4880-93a9-46b26fc5a65c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca95f9666c62498dbaaea4aeafb4a152, user_id=c32582762c9946e29f32c7ff35260aa5, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:15.082890 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03312cf3-334c-4880-93a9-46b26fc5a65c admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:15.098245 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 982/1988] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:21:15 2022] PUT /identity/v3/projects/ca95f9666c62498dbaaea4aeafb4a152/users/c32582762c9946e29f32c7ff35260aa5/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:21:15.109718 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a7eb05-25ce-4d20-b227-f85768f9ded6 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:15.110069 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a7eb05-25ce-4d20-b227-f85768f9ded6 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:15.110407 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a7eb05-25ce-4d20-b227-f85768f9ded6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:15.139407 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-08a7eb05-25ce-4d20-b227-f85768f9ded6 None None] MFA Rules not processed for user `c32582762c9946e29f32c7ff35260aa5`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:15.140649 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-08a7eb05-25ce-4d20-b227-f85768f9ded6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:15.182546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03312cf3-334c-4880-93a9-46b26fc5a65c admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:15.184693 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-03312cf3-334c-4880-93a9-46b26fc5a65c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:15.238780 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1007/1989] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:15 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:21:15.252149 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a7eb05-25ce-4d20-b227-f85768f9ded6 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:15.253722 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-08a7eb05-25ce-4d20-b227-f85768f9ded6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:15.277945 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:15.304824 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03312cf3-334c-4880-93a9-46b26fc5a65c admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:15.313590 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ec0a9-9d54-4769-875c-9ff19352988d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:15.313590 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ec0a9-9d54-4769-875c-9ff19352988d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:15.313590 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ec0a9-9d54-4769-875c-9ff19352988d admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:15.313590 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ec0a9-9d54-4769-875c-9ff19352988d admin admin] PATH_INFO: `/v3/projects/3df4b5b5f690447b8638e036bb602f99` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:15.314920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24ec0a9-9d54-4769-875c-9ff19352988d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3df4b5b5f690447b8638e036bb602f99)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:15.316990 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24ec0a9-9d54-4769-875c-9ff19352988d admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:15.331446 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a7eb05-25ce-4d20-b227-f85768f9ded6 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:15.335588 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5190b1-27d2-430f-8770-7bf363d7048a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:15.336299 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5190b1-27d2-430f-8770-7bf363d7048a service cinder] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:15.336588 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5190b1-27d2-430f-8770-7bf363d7048a service cinder] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:15.339645 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5190b1-27d2-430f-8770-7bf363d7048a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:15.350517 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5190b1-27d2-430f-8770-7bf363d7048a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:15.352126 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5190b1-27d2-430f-8770-7bf363d7048a service cinder] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:15.362614 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1008/1990] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:21:15 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:21:15.437626 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-a24ec0a9-9d54-4769-875c-9ff19352988d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3df4b5b5f690447b8638e036bb602f99', 'request_id': 'req-a24ec0a9-9d54-4769-875c-9ff19352988d'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:15.443955 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-a24ec0a9-9d54-4769-875c-9ff19352988d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3df4b5b5f690447b8638e036bb602f99', 'request_id': 'req-a24ec0a9-9d54-4769-875c-9ff19352988d'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:15.541079 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-a24ec0a9-9d54-4769-875c-9ff19352988d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3df4b5b5f690447b8638e036bb602f99', 'request_id': 'req-a24ec0a9-9d54-4769-875c-9ff19352988d'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:15.555579 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-a24ec0a9-9d54-4769-875c-9ff19352988d admin admin] The token cache is being invalidate because project 3df4b5b5f690447b8638e036bb602f99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:21:15.555939 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-a24ec0a9-9d54-4769-875c-9ff19352988d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:15.556634 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-a24ec0a9-9d54-4769-875c-9ff19352988d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:15.558510 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 983/1991] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:21:15 2022] DELETE /identity/v3/projects/3df4b5b5f690447b8638e036bb602f99 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:15.562935 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5190b1-27d2-430f-8770-7bf363d7048a service cinder] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:15.576064 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5190b1-27d2-430f-8770-7bf363d7048a service cinder] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:15.618268 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ec0a9-9d54-4769-875c-9ff19352988d admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:15.619990 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-a24ec0a9-9d54-4769-875c-9ff19352988d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:15.647461 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300526ab-a459-4048-afa2-df13b016f08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:15.648662 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300526ab-a459-4048-afa2-df13b016f08d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:15.648953 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300526ab-a459-4048-afa2-df13b016f08d admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:15.649228 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300526ab-a459-4048-afa2-df13b016f08d admin admin] PATH_INFO: `/v3/users/49427c7a7ba74cea9f4d4c76635ae98b` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:15.650396 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300526ab-a459-4048-afa2-df13b016f08d admin admin] RBAC: Authorizing `identity:delete_user(user_id=49427c7a7ba74cea9f4d4c76635ae98b)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:15.652690 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300526ab-a459-4048-afa2-df13b016f08d admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:15.690286 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ec0a9-9d54-4769-875c-9ff19352988d admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:15.694207 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414113ab-589c-4444-859f-66ae8934bb9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:15.694701 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414113ab-589c-4444-859f-66ae8934bb9e service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:15.694987 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414113ab-589c-4444-859f-66ae8934bb9e service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:15.695277 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414113ab-589c-4444-859f-66ae8934bb9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:15.698416 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-414113ab-589c-4444-859f-66ae8934bb9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:15.777762 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414113ab-589c-4444-859f-66ae8934bb9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:15.779303 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414113ab-589c-4444-859f-66ae8934bb9e service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:15.797885 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 984/1992] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:21:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:21:15.861821 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414113ab-589c-4444-859f-66ae8934bb9e service cinder] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:15.861821 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-414113ab-589c-4444-859f-66ae8934bb9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:15.872601 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414113ab-589c-4444-859f-66ae8934bb9e service cinder] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:15.877184 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5fec7a-c49a-4f7e-8065-bb0ac94a867e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:15.878037 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5fec7a-c49a-4f7e-8065-bb0ac94a867e service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:15.878504 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5fec7a-c49a-4f7e-8065-bb0ac94a867e service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:15.880837 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5fec7a-c49a-4f7e-8065-bb0ac94a867e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:15.883865 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-8d5fec7a-c49a-4f7e-8065-bb0ac94a867e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:15.900159 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-300526ab-a459-4048-afa2-df13b016f08d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49427c7a7ba74cea9f4d4c76635ae98b'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:15.907726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-300526ab-a459-4048-afa2-df13b016f08d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49427c7a7ba74cea9f4d4c76635ae98b'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:15.908734 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-300526ab-a459-4048-afa2-df13b016f08d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49427c7a7ba74cea9f4d4c76635ae98b'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:15.923871 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-300526ab-a459-4048-afa2-df13b016f08d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49427c7a7ba74cea9f4d4c76635ae98b'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:15.952751 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5fec7a-c49a-4f7e-8065-bb0ac94a867e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:15.954203 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5fec7a-c49a-4f7e-8065-bb0ac94a867e service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:15.963860 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 985/1993] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:21:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:21:16.031614 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-300526ab-a459-4048-afa2-df13b016f08d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49427c7a7ba74cea9f4d4c76635ae98b'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:16.038681 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1009/1994] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:21:15 2022] DELETE /identity/v3/users/49427c7a7ba74cea9f4d4c76635ae98b => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:16.346409 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:16.517314 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:16.988267 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5fec7a-c49a-4f7e-8065-bb0ac94a867e service cinder] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:16.990926 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-8d5fec7a-c49a-4f7e-8065-bb0ac94a867e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:17.010379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5fec7a-c49a-4f7e-8065-bb0ac94a867e service cinder] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:17.115276 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527141f-abdb-4644-87fa-eec863d012a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:17.116057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527141f-abdb-4644-87fa-eec863d012a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:17.116378 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527141f-abdb-4644-87fa-eec863d012a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:17.116644 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527141f-abdb-4644-87fa-eec863d012a9 admin admin] PATH_INFO: `/v3/projects/cba4a46eeef84dc4940b41d2a93e2a3d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:17.118457 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527141f-abdb-4644-87fa-eec863d012a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cba4a46eeef84dc4940b41d2a93e2a3d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:17.120198 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527141f-abdb-4644-87fa-eec863d012a9 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:17.279512 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-c527141f-abdb-4644-87fa-eec863d012a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cba4a46eeef84dc4940b41d2a93e2a3d', 'request_id': 'req-c527141f-abdb-4644-87fa-eec863d012a9'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:17.296705 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-c527141f-abdb-4644-87fa-eec863d012a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cba4a46eeef84dc4940b41d2a93e2a3d', 'request_id': 'req-c527141f-abdb-4644-87fa-eec863d012a9'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:17.408200 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-c527141f-abdb-4644-87fa-eec863d012a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cba4a46eeef84dc4940b41d2a93e2a3d', 'request_id': 'req-c527141f-abdb-4644-87fa-eec863d012a9'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:17.426082 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-c527141f-abdb-4644-87fa-eec863d012a9 admin admin] The token cache is being invalidate because project cba4a46eeef84dc4940b41d2a93e2a3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:21:17.426507 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-c527141f-abdb-4644-87fa-eec863d012a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:17.427286 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-c527141f-abdb-4644-87fa-eec863d012a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:17.430696 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 986/1995] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:21:16 2022] DELETE /identity/v3/projects/cba4a46eeef84dc4940b41d2a93e2a3d => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:17.439675 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c407b-9287-42b9-a083-da3976d4fbd0 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:17.440205 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c407b-9287-42b9-a083-da3976d4fbd0 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:17.440296 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c407b-9287-42b9-a083-da3976d4fbd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:17.463019 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-117c407b-9287-42b9-a083-da3976d4fbd0 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:17.464633 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-117c407b-9287-42b9-a083-da3976d4fbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:17.544746 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1010/1996] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:21:17.554256 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc3f19-310c-468a-a08d-31b17748371a None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:17.554563 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc3f19-310c-468a-a08d-31b17748371a None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:17.554844 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc3f19-310c-468a-a08d-31b17748371a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:17.577037 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-8fcc3f19-310c-468a-a08d-31b17748371a None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:17.578325 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-8fcc3f19-310c-468a-a08d-31b17748371a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:17.586959 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 987/1997] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:17.591162 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c407b-9287-42b9-a083-da3976d4fbd0 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:17.601230 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c407b-9287-42b9-a083-da3976d4fbd0 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:17.605867 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0add4b-942b-44e3-8193-0bfc46785723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:17.606777 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0add4b-942b-44e3-8193-0bfc46785723 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:17.607062 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0add4b-942b-44e3-8193-0bfc46785723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:17.607341 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0add4b-942b-44e3-8193-0bfc46785723 admin admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:17.608592 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0add4b-942b-44e3-8193-0bfc46785723 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:21:17.608900 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0add4b-942b-44e3-8193-0bfc46785723 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:17.610551 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0add4b-942b-44e3-8193-0bfc46785723 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:17.623471 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1011/1998] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:21:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:21:17.628455 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcc3f19-310c-468a-a08d-31b17748371a None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:17.637873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcc3f19-310c-468a-a08d-31b17748371a None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:17.642800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b006b573-475d-4f05-8269-13046331c2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:17.647594 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b006b573-475d-4f05-8269-13046331c2e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:17.647594 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b006b573-475d-4f05-8269-13046331c2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:17.647594 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b006b573-475d-4f05-8269-13046331c2e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:17.647594 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b006b573-475d-4f05-8269-13046331c2e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:17.647594 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b006b573-475d-4f05-8269-13046331c2e2 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:17.785288 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 988/1999] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:21:17 2022] POST /identity/v3/projects => generated 385 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:21:17.786566 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0add4b-942b-44e3-8193-0bfc46785723 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:17.797825 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0add4b-942b-44e3-8193-0bfc46785723 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:17.890778 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d40c1-b9c3-49a4-8492-e9fd5c8c38cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:17.892022 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d40c1-b9c3-49a4-8492-e9fd5c8c38cb admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:17.892022 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d40c1-b9c3-49a4-8492-e9fd5c8c38cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:17.892022 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d40c1-b9c3-49a4-8492-e9fd5c8c38cb admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:17.892460 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13d40c1-b9c3-49a4-8492-e9fd5c8c38cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:17.895354 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13d40c1-b9c3-49a4-8492-e9fd5c8c38cb admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:17.998301 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b006b573-475d-4f05-8269-13046331c2e2 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:18.000640 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-b006b573-475d-4f05-8269-13046331c2e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:18.007775 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1012/2000] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:21:17 2022] POST /identity/v3/users => generated 293 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:21:18.013530 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d40c1-b9c3-49a4-8492-e9fd5c8c38cb admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:18.029261 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d40c1-b9c3-49a4-8492-e9fd5c8c38cb admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:18.032642 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633223d3-c41e-4adc-ab59-c77469151caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:18.033276 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633223d3-c41e-4adc-ab59-c77469151caa admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:18.033566 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633223d3-c41e-4adc-ab59-c77469151caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:18.033967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633223d3-c41e-4adc-ab59-c77469151caa admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:18.034637 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633223d3-c41e-4adc-ab59-c77469151caa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:18.036421 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633223d3-c41e-4adc-ab59-c77469151caa admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:18.050809 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1013/2001] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 12:21:18.053609 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633223d3-c41e-4adc-ab59-c77469151caa admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:18.071508 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b006b573-475d-4f05-8269-13046331c2e2 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:18.075248 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633223d3-c41e-4adc-ab59-c77469151caa admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:18.077839 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f76c6-cafc-4f9b-97b4-3d5dbc300c3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:18.078879 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f76c6-cafc-4f9b-97b4-3d5dbc300c3f service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:18.085822 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f76c6-cafc-4f9b-97b4-3d5dbc300c3f service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:18.085822 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f76c6-cafc-4f9b-97b4-3d5dbc300c3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:18.085963 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf9dc93-ff23-4b36-b731-241e6a38901a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:18.085963 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9dc93-ff23-4b36-b731-241e6a38901a admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:18.085963 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9dc93-ff23-4b36-b731-241e6a38901a admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:18.085963 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9dc93-ff23-4b36-b731-241e6a38901a admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:18.086230 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf9dc93-ff23-4b36-b731-241e6a38901a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:18.088571 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf9dc93-ff23-4b36-b731-241e6a38901a admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:18.089864 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-0f3f76c6-cafc-4f9b-97b4-3d5dbc300c3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:18.109398 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1014/2002] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 12:21:18.115486 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf9dc93-ff23-4b36-b731-241e6a38901a admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:18.135207 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf9dc93-ff23-4b36-b731-241e6a38901a admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:18.142492 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb46caea-1ea6-4a68-ab49-ac1d982f4ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:18.143204 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46caea-1ea6-4a68-ab49-ac1d982f4ac4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:18.143488 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46caea-1ea6-4a68-ab49-ac1d982f4ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:18.148690 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46caea-1ea6-4a68-ab49-ac1d982f4ac4 admin admin] PATH_INFO: `/v3/projects/d9f6439b977648dab4c41480d69a0340/users/ac825664b4324c8da293b2aa9c3a9f9e/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:18.164572 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb46caea-1ea6-4a68-ab49-ac1d982f4ac4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9f6439b977648dab4c41480d69a0340, user_id=ac825664b4324c8da293b2aa9c3a9f9e, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:18.168591 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb46caea-1ea6-4a68-ab49-ac1d982f4ac4 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:18.169709 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:18.200117 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f76c6-cafc-4f9b-97b4-3d5dbc300c3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:18.201575 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f76c6-cafc-4f9b-97b4-3d5dbc300c3f service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:18.213147 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 989/2003] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:21:17 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:21:18.301201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1015/2004] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:21:18 2022] PUT /identity/v3/projects/d9f6439b977648dab4c41480d69a0340/users/ac825664b4324c8da293b2aa9c3a9f9e/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:18.312198 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74af01-9b17-40cf-8bc5-b56210bc2eb0 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:18.312483 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74af01-9b17-40cf-8bc5-b56210bc2eb0 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:18.313538 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74af01-9b17-40cf-8bc5-b56210bc2eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:18.343775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-6d74af01-9b17-40cf-8bc5-b56210bc2eb0 None None] MFA Rules not processed for user `ac825664b4324c8da293b2aa9c3a9f9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:18.345105 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-6d74af01-9b17-40cf-8bc5-b56210bc2eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:18.438295 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 990/2005] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:18.442908 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb46caea-1ea6-4a68-ab49-ac1d982f4ac4 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:18.538906 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb46caea-1ea6-4a68-ab49-ac1d982f4ac4 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:18.549710 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b0fb38-ac8a-4f61-b915-c7340c1e53d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:18.550070 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b0fb38-ac8a-4f61-b915-c7340c1e53d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:18.550326 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b0fb38-ac8a-4f61-b915-c7340c1e53d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:18.552555 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b0fb38-ac8a-4f61-b915-c7340c1e53d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:18.553183 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b0fb38-ac8a-4f61-b915-c7340c1e53d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:18.558203 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b0fb38-ac8a-4f61-b915-c7340c1e53d9 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:18.788128 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1016/2006] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:21:18 2022] POST /identity/v3/projects => generated 387 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:21:18.792444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d74af01-9b17-40cf-8bc5-b56210bc2eb0 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:18.812831 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d74af01-9b17-40cf-8bc5-b56210bc2eb0 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:18.895849 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abda6f8-f0a7-45da-baab-b7e61829282a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:18.896876 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abda6f8-f0a7-45da-baab-b7e61829282a admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:18.907525 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abda6f8-f0a7-45da-baab-b7e61829282a admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:18.907525 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abda6f8-f0a7-45da-baab-b7e61829282a admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:18.907525 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abda6f8-f0a7-45da-baab-b7e61829282a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:18.907525 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abda6f8-f0a7-45da-baab-b7e61829282a admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:18.961748 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:19.036630 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 991/2007] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:21:18 2022] POST /identity/v3/users => generated 294 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:21:19.041646 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b0fb38-ac8a-4f61-b915-c7340c1e53d9 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:19.053740 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b0fb38-ac8a-4f61-b915-c7340c1e53d9 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:19.058588 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e131555-5c91-483b-a664-e43082462dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:19.059202 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e131555-5c91-483b-a664-e43082462dac admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:19.059511 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e131555-5c91-483b-a664-e43082462dac admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:19.059789 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e131555-5c91-483b-a664-e43082462dac admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:19.060441 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e131555-5c91-483b-a664-e43082462dac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:19.062678 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e131555-5c91-483b-a664-e43082462dac admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:19.084073 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1017/2008] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 12:21:19.084304 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abda6f8-f0a7-45da-baab-b7e61829282a admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:19.089811 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abda6f8-f0a7-45da-baab-b7e61829282a admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:19.095181 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f767b21-9b95-4c88-83ae-8138a90838a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:19.095860 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f767b21-9b95-4c88-83ae-8138a90838a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:19.096131 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f767b21-9b95-4c88-83ae-8138a90838a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:19.096409 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f767b21-9b95-4c88-83ae-8138a90838a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:19.097071 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f767b21-9b95-4c88-83ae-8138a90838a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:19.098845 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f767b21-9b95-4c88-83ae-8138a90838a0 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:19.110237 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 992/2009] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:21:19.115263 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e131555-5c91-483b-a664-e43082462dac admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:19.127627 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e131555-5c91-483b-a664-e43082462dac admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:19.132392 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3bbafe-5d49-4105-b242-86f87116958d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:19.133057 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bbafe-5d49-4105-b242-86f87116958d admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:19.133583 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bbafe-5d49-4105-b242-86f87116958d admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:19.133873 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bbafe-5d49-4105-b242-86f87116958d admin admin] PATH_INFO: `/v3/projects/23de3c57d61244adbb7228a3f9a82223/users/0e1e758a21484fc28f31dbc830fc5ee4/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:19.149148 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3bbafe-5d49-4105-b242-86f87116958d admin admin] RBAC: Authorizing `identity:create_grant(project_id=23de3c57d61244adbb7228a3f9a82223, user_id=0e1e758a21484fc28f31dbc830fc5ee4, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:19.154888 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3bbafe-5d49-4105-b242-86f87116958d admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:19.300712 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1018/2010] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:21:19 2022] PUT /identity/v3/projects/23de3c57d61244adbb7228a3f9a82223/users/0e1e758a21484fc28f31dbc830fc5ee4/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:19.313942 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4ffda-bd59-400c-b03b-a907f2829dc6 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:19.314237 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4ffda-bd59-400c-b03b-a907f2829dc6 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:19.314556 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4ffda-bd59-400c-b03b-a907f2829dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:19.345179 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-f5d4ffda-bd59-400c-b03b-a907f2829dc6 None None] MFA Rules not processed for user `0e1e758a21484fc28f31dbc830fc5ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:19.346356 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-f5d4ffda-bd59-400c-b03b-a907f2829dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:19.423770 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 993/2011] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:19 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:19.428049 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3bbafe-5d49-4105-b242-86f87116958d admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:19.492356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3bbafe-5d49-4105-b242-86f87116958d admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:19.499045 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b6aed-2bec-4441-ac6a-8f04b733f28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:19.499892 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b6aed-2bec-4441-ac6a-8f04b733f28c admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:19.500373 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b6aed-2bec-4441-ac6a-8f04b733f28c admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:19.500628 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b6aed-2bec-4441-ac6a-8f04b733f28c admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:19.501453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108b6aed-2bec-4441-ac6a-8f04b733f28c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:19.503936 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108b6aed-2bec-4441-ac6a-8f04b733f28c admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:19.587239 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1019/2012] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:21:19 2022] POST /identity/v3/projects => generated 387 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:21:19.591451 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4ffda-bd59-400c-b03b-a907f2829dc6 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:19.604609 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4ffda-bd59-400c-b03b-a907f2829dc6 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:19.682368 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6316e0cc-6f3c-446d-940c-cca4b640cb55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:19.683196 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6316e0cc-6f3c-446d-940c-cca4b640cb55 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:19.683549 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6316e0cc-6f3c-446d-940c-cca4b640cb55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:19.683849 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6316e0cc-6f3c-446d-940c-cca4b640cb55 admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:19.684505 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6316e0cc-6f3c-446d-940c-cca4b640cb55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:19.686513 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6316e0cc-6f3c-446d-940c-cca4b640cb55 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:19.787732 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 994/2013] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:21:19 2022] POST /identity/v3/users => generated 300 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:21:19.791727 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b6aed-2bec-4441-ac6a-8f04b733f28c admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:19.802304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b6aed-2bec-4441-ac6a-8f04b733f28c admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:19.807019 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d361f2d-270f-41c8-bdc8-818103ad7e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:19.807657 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d361f2d-270f-41c8-bdc8-818103ad7e0e admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:19.807946 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d361f2d-270f-41c8-bdc8-818103ad7e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:19.808228 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d361f2d-270f-41c8-bdc8-818103ad7e0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:19.808883 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d361f2d-270f-41c8-bdc8-818103ad7e0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:19.810490 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d361f2d-270f-41c8-bdc8-818103ad7e0e admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:19.822711 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1020/2014] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:21:19.827626 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6316e0cc-6f3c-446d-940c-cca4b640cb55 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:19.838182 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6316e0cc-6f3c-446d-940c-cca4b640cb55 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:19.843239 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807611c8-9ace-4399-a180-392f6ca257cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:19.843958 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807611c8-9ace-4399-a180-392f6ca257cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:19.844216 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807611c8-9ace-4399-a180-392f6ca257cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:19.844498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807611c8-9ace-4399-a180-392f6ca257cb admin admin] PATH_INFO: `/v3/projects/fb0b05729efd4000bd504bc5779d1366/users/0fb6fae0770d42839ee8ff0330faf1ff/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:19.862697 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807611c8-9ace-4399-a180-392f6ca257cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb0b05729efd4000bd504bc5779d1366, user_id=0fb6fae0770d42839ee8ff0330faf1ff, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:19.865423 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807611c8-9ace-4399-a180-392f6ca257cb admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:19.961382 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 995/2015] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:21:19 2022] PUT /identity/v3/projects/fb0b05729efd4000bd504bc5779d1366/users/0fb6fae0770d42839ee8ff0330faf1ff/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:19.966557 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d361f2d-270f-41c8-bdc8-818103ad7e0e admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:19.995135 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d361f2d-270f-41c8-bdc8-818103ad7e0e admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:20.080548 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43270666-9ab9-47a4-82f2-86298044067a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:20.081164 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43270666-9ab9-47a4-82f2-86298044067a admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:20.081463 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43270666-9ab9-47a4-82f2-86298044067a admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:20.081727 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43270666-9ab9-47a4-82f2-86298044067a admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:20.082372 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43270666-9ab9-47a4-82f2-86298044067a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:20.085810 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43270666-9ab9-47a4-82f2-86298044067a admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:20.095527 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1021/2016] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:21:20.109239 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807611c8-9ace-4399-a180-392f6ca257cb admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:20.120040 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807611c8-9ace-4399-a180-392f6ca257cb admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:20.126377 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c46ffc9-0565-44f1-bd32-74143ed69cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:20.126600 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46ffc9-0565-44f1-bd32-74143ed69cce admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:20.126932 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46ffc9-0565-44f1-bd32-74143ed69cce admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:20.127118 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46ffc9-0565-44f1-bd32-74143ed69cce admin admin] PATH_INFO: `/v3/projects/fb0b05729efd4000bd504bc5779d1366/users/0fb6fae0770d42839ee8ff0330faf1ff/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:20.130526 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c46ffc9-0565-44f1-bd32-74143ed69cce admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb0b05729efd4000bd504bc5779d1366, user_id=0fb6fae0770d42839ee8ff0330faf1ff, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:20.133883 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c46ffc9-0565-44f1-bd32-74143ed69cce admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:20.146894 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 996/2017] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:21:20 2022] PUT /identity/v3/projects/fb0b05729efd4000bd504bc5779d1366/users/0fb6fae0770d42839ee8ff0330faf1ff/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:20.153222 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:20.158851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518312c3-9f99-402a-b9d8-6ea1ba87dbf1 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:20.159155 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518312c3-9f99-402a-b9d8-6ea1ba87dbf1 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:20.159443 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518312c3-9f99-402a-b9d8-6ea1ba87dbf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:20.192171 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-518312c3-9f99-402a-b9d8-6ea1ba87dbf1 None None] MFA Rules not processed for user `0fb6fae0770d42839ee8ff0330faf1ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:20.193400 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-518312c3-9f99-402a-b9d8-6ea1ba87dbf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:20.332024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1022/2018] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:20 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:20.332024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:20.472131 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c46ffc9-0565-44f1-bd32-74143ed69cce admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:20.473890 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-2c46ffc9-0565-44f1-bd32-74143ed69cce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:20.533055 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c46ffc9-0565-44f1-bd32-74143ed69cce admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:20.537383 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982568be-cfdc-435b-bc32-b2546184f209 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:20.538033 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982568be-cfdc-435b-bc32-b2546184f209 service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:20.538318 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982568be-cfdc-435b-bc32-b2546184f209 service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:20.538593 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982568be-cfdc-435b-bc32-b2546184f209 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:20.550757 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982568be-cfdc-435b-bc32-b2546184f209 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:20.552290 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982568be-cfdc-435b-bc32-b2546184f209 service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:20.563343 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 997/2019] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:21:20 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:21:21.288203 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:21.601993 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:22.460448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:22.675823 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:22.880433 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518312c3-9f99-402a-b9d8-6ea1ba87dbf1 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:22.901174 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518312c3-9f99-402a-b9d8-6ea1ba87dbf1 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:22.906067 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4db8a-152d-441a-bc9e-820b2b80572c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:22.908365 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4db8a-152d-441a-bc9e-820b2b80572c service cinder] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:22.908365 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4db8a-152d-441a-bc9e-820b2b80572c service cinder] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:22.908365 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4db8a-152d-441a-bc9e-820b2b80572c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:22.968705 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c4db8a-152d-441a-bc9e-820b2b80572c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:22.970197 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c4db8a-152d-441a-bc9e-820b2b80572c service cinder] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:22.979552 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1023/2020] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:21:22 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:21:23.375202 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:24.001680 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:24.333053 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:24.931355 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982568be-cfdc-435b-bc32-b2546184f209 service cinder] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:24.934408 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-982568be-cfdc-435b-bc32-b2546184f209 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:24.980614 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:25.009574 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982568be-cfdc-435b-bc32-b2546184f209 service cinder] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:25.014806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8357c5-7b07-493f-a471-b7971eff7fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:25.015500 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8357c5-7b07-493f-a471-b7971eff7fc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:25.015783 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8357c5-7b07-493f-a471-b7971eff7fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:25.016060 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8357c5-7b07-493f-a471-b7971eff7fc8 admin admin] PATH_INFO: `/v3/users/5dce9587082a435093b849e4a712f2a3` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:25.017186 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8357c5-7b07-493f-a471-b7971eff7fc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dce9587082a435093b849e4a712f2a3)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:25.019135 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8357c5-7b07-493f-a471-b7971eff7fc8 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:25.134225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:25.204779 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-3f8357c5-7b07-493f-a471-b7971eff7fc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dce9587082a435093b849e4a712f2a3'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:25.205449 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-3f8357c5-7b07-493f-a471-b7971eff7fc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dce9587082a435093b849e4a712f2a3'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:25.206607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-3f8357c5-7b07-493f-a471-b7971eff7fc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dce9587082a435093b849e4a712f2a3'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:25.213210 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-3f8357c5-7b07-493f-a471-b7971eff7fc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dce9587082a435093b849e4a712f2a3'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:25.225001 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-3f8357c5-7b07-493f-a471-b7971eff7fc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dce9587082a435093b849e4a712f2a3'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:25.300398 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 998/2021] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:21:24 2022] DELETE /identity/v3/users/5dce9587082a435093b849e4a712f2a3 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:25.306033 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f01c9-6648-4727-8c85-1bdd1d85adbf None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:25.306503 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f01c9-6648-4727-8c85-1bdd1d85adbf None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:25.306594 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f01c9-6648-4727-8c85-1bdd1d85adbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:25.325705 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-917f01c9-6648-4727-8c85-1bdd1d85adbf None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:25.326845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-917f01c9-6648-4727-8c85-1bdd1d85adbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:25.388570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1024/2022] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:21:25.398373 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8357c5-7b07-493f-a471-b7971eff7fc8 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:25.399805 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-3f8357c5-7b07-493f-a471-b7971eff7fc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:25.460730 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8357c5-7b07-493f-a471-b7971eff7fc8 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:25.464455 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb80bef-0e3f-428a-8fc4-968d80962955 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:25.465142 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb80bef-0e3f-428a-8fc4-968d80962955 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:25.465422 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb80bef-0e3f-428a-8fc4-968d80962955 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:25.465692 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb80bef-0e3f-428a-8fc4-968d80962955 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:25.479615 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb80bef-0e3f-428a-8fc4-968d80962955 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:25.481158 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb80bef-0e3f-428a-8fc4-968d80962955 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:25.491844 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 999/2023] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:21:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:21:25.819790 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:INSERT=6,keystone:SELECT=87,keystone:DELETE=7 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:26.003178 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f01c9-6648-4727-8c85-1bdd1d85adbf None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:26.005478 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-917f01c9-6648-4727-8c85-1bdd1d85adbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:26.006912 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:26.020241 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f01c9-6648-4727-8c85-1bdd1d85adbf None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:26.025041 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd943418-3b91-46a5-9d17-ba606d7c8680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:26.025676 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd943418-3b91-46a5-9d17-ba606d7c8680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:26.025962 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd943418-3b91-46a5-9d17-ba606d7c8680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:26.026242 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd943418-3b91-46a5-9d17-ba606d7c8680 admin admin] PATH_INFO: `/v3/projects/5652962ee951443a8d432bb46fe1fd8a` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:26.027373 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd943418-3b91-46a5-9d17-ba606d7c8680 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5652962ee951443a8d432bb46fe1fd8a)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:26.029352 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd943418-3b91-46a5-9d17-ba606d7c8680 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:26.133632 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-cd943418-3b91-46a5-9d17-ba606d7c8680 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5652962ee951443a8d432bb46fe1fd8a', 'request_id': 'req-cd943418-3b91-46a5-9d17-ba606d7c8680'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:26.140320 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-cd943418-3b91-46a5-9d17-ba606d7c8680 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5652962ee951443a8d432bb46fe1fd8a', 'request_id': 'req-cd943418-3b91-46a5-9d17-ba606d7c8680'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:26.225197 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-cd943418-3b91-46a5-9d17-ba606d7c8680 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5652962ee951443a8d432bb46fe1fd8a', 'request_id': 'req-cd943418-3b91-46a5-9d17-ba606d7c8680'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:26.237217 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-cd943418-3b91-46a5-9d17-ba606d7c8680 admin admin] The token cache is being invalidate because project 5652962ee951443a8d432bb46fe1fd8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:21:26.237509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-cd943418-3b91-46a5-9d17-ba606d7c8680 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:26.238135 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-cd943418-3b91-46a5-9d17-ba606d7c8680 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:26.239640 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1025/2024] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:21:26 2022] DELETE /identity/v3/projects/5652962ee951443a8d432bb46fe1fd8a => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:26.243662 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb80bef-0e3f-428a-8fc4-968d80962955 service neutron] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:26.254096 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:26.254679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb80bef-0e3f-428a-8fc4-968d80962955 service neutron] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:26.312097 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a603ab3-464b-4e1b-9262-2b51a6544fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:26.312737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a603ab3-464b-4e1b-9262-2b51a6544fca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:26.313023 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a603ab3-464b-4e1b-9262-2b51a6544fca admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:26.313590 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a603ab3-464b-4e1b-9262-2b51a6544fca admin admin] PATH_INFO: `/v3/users/40984d4b52154ccf912552adf98ecdb3` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:26.314488 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a603ab3-464b-4e1b-9262-2b51a6544fca admin admin] RBAC: Authorizing `identity:delete_user(user_id=40984d4b52154ccf912552adf98ecdb3)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:26.316425 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a603ab3-464b-4e1b-9262-2b51a6544fca admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:26.417911 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:26.500547 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-7a603ab3-464b-4e1b-9262-2b51a6544fca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40984d4b52154ccf912552adf98ecdb3'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:26.501334 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-7a603ab3-464b-4e1b-9262-2b51a6544fca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40984d4b52154ccf912552adf98ecdb3'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:26.502008 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-7a603ab3-464b-4e1b-9262-2b51a6544fca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40984d4b52154ccf912552adf98ecdb3'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:26.509149 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-7a603ab3-464b-4e1b-9262-2b51a6544fca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40984d4b52154ccf912552adf98ecdb3'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:26.520710 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-7a603ab3-464b-4e1b-9262-2b51a6544fca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40984d4b52154ccf912552adf98ecdb3'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:26.577855 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:26.614070 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1000/2025] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:21:26 2022] DELETE /identity/v3/users/40984d4b52154ccf912552adf98ecdb3 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:26.848406 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:27.007658 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd943418-3b91-46a5-9d17-ba606d7c8680 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:27.009216 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-cd943418-3b91-46a5-9d17-ba606d7c8680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:27.023237 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd943418-3b91-46a5-9d17-ba606d7c8680 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:27.025944 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:27.084216 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b942c-9775-4c72-801f-8b229d5b436c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:27.084859 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b942c-9775-4c72-801f-8b229d5b436c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:27.085146 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b942c-9775-4c72-801f-8b229d5b436c admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:27.085427 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b942c-9775-4c72-801f-8b229d5b436c admin admin] PATH_INFO: `/v3/projects/973cabad6bba47fd9628b80ef05771eb` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:27.086565 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685b942c-9775-4c72-801f-8b229d5b436c admin admin] RBAC: Authorizing `identity:delete_project(project_id=973cabad6bba47fd9628b80ef05771eb)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:27.088553 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685b942c-9775-4c72-801f-8b229d5b436c admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:27.245737 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-685b942c-9775-4c72-801f-8b229d5b436c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '973cabad6bba47fd9628b80ef05771eb', 'request_id': 'req-685b942c-9775-4c72-801f-8b229d5b436c'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:27.248339 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-685b942c-9775-4c72-801f-8b229d5b436c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '973cabad6bba47fd9628b80ef05771eb', 'request_id': 'req-685b942c-9775-4c72-801f-8b229d5b436c'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:27.264250 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:27.356584 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-685b942c-9775-4c72-801f-8b229d5b436c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '973cabad6bba47fd9628b80ef05771eb', 'request_id': 'req-685b942c-9775-4c72-801f-8b229d5b436c'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:27.369059 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-685b942c-9775-4c72-801f-8b229d5b436c admin admin] The token cache is being invalidate because project 973cabad6bba47fd9628b80ef05771eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:21:27.369341 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-685b942c-9775-4c72-801f-8b229d5b436c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:27.370003 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-685b942c-9775-4c72-801f-8b229d5b436c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:27.371795 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1026/2026] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:21:27 2022] DELETE /identity/v3/projects/973cabad6bba47fd9628b80ef05771eb => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:27.376133 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a603ab3-464b-4e1b-9262-2b51a6544fca admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:27.377613 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-7a603ab3-464b-4e1b-9262-2b51a6544fca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:27.390983 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a603ab3-464b-4e1b-9262-2b51a6544fca admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:27.391529 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:27.451682 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67edab5-ab0c-46ac-ac68-18c1ffd19a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:27.453193 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67edab5-ab0c-46ac-ac68-18c1ffd19a52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:27.453581 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67edab5-ab0c-46ac-ac68-18c1ffd19a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:27.453947 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67edab5-ab0c-46ac-ac68-18c1ffd19a52 admin admin] PATH_INFO: `/v3/users/6b620f862ebb4e62bf90fd0ccb9e281c` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:27.455475 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67edab5-ab0c-46ac-ac68-18c1ffd19a52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b620f862ebb4e62bf90fd0ccb9e281c)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:27.457814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67edab5-ab0c-46ac-ac68-18c1ffd19a52 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:27.596915 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:27.689566 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-e67edab5-ab0c-46ac-ac68-18c1ffd19a52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b620f862ebb4e62bf90fd0ccb9e281c'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:27.690296 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-e67edab5-ab0c-46ac-ac68-18c1ffd19a52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b620f862ebb4e62bf90fd0ccb9e281c'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:27.690911 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-e67edab5-ab0c-46ac-ac68-18c1ffd19a52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b620f862ebb4e62bf90fd0ccb9e281c'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:27.698139 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-e67edab5-ab0c-46ac-ac68-18c1ffd19a52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b620f862ebb4e62bf90fd0ccb9e281c'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:27.714102 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-e67edab5-ab0c-46ac-ac68-18c1ffd19a52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b620f862ebb4e62bf90fd0ccb9e281c'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:27.808375 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1001/2027] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:21:27 2022] DELETE /identity/v3/users/6b620f862ebb4e62bf90fd0ccb9e281c => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:28.122960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:28.266879 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b942c-9775-4c72-801f-8b229d5b436c admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:28.268504 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-685b942c-9775-4c72-801f-8b229d5b436c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:28.278163 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:28.301034 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b942c-9775-4c72-801f-8b229d5b436c admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:28.368122 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e67aaa-0911-4538-a37a-870dc6a08b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:28.368772 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e67aaa-0911-4538-a37a-870dc6a08b5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:28.369063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e67aaa-0911-4538-a37a-870dc6a08b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:28.369382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e67aaa-0911-4538-a37a-870dc6a08b5b admin admin] PATH_INFO: `/v3/projects/f7cf769018dc4d268ca94cdc49c9c037` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:28.371321 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e67aaa-0911-4538-a37a-870dc6a08b5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7cf769018dc4d268ca94cdc49c9c037)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:28.373607 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e67aaa-0911-4538-a37a-870dc6a08b5b admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:28.538506 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-10e67aaa-0911-4538-a37a-870dc6a08b5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7cf769018dc4d268ca94cdc49c9c037', 'request_id': 'req-10e67aaa-0911-4538-a37a-870dc6a08b5b'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:28.546678 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-10e67aaa-0911-4538-a37a-870dc6a08b5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7cf769018dc4d268ca94cdc49c9c037', 'request_id': 'req-10e67aaa-0911-4538-a37a-870dc6a08b5b'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:28.623231 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-10e67aaa-0911-4538-a37a-870dc6a08b5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7cf769018dc4d268ca94cdc49c9c037', 'request_id': 'req-10e67aaa-0911-4538-a37a-870dc6a08b5b'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:28.635580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-10e67aaa-0911-4538-a37a-870dc6a08b5b admin admin] The token cache is being invalidate because project f7cf769018dc4d268ca94cdc49c9c037 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:21:28.635796 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-10e67aaa-0911-4538-a37a-870dc6a08b5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:28.636445 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-10e67aaa-0911-4538-a37a-870dc6a08b5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:28.637987 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1027/2028] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:21:28 2022] DELETE /identity/v3/projects/f7cf769018dc4d268ca94cdc49c9c037 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:28.643092 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e6cd3d-f199-405c-8f4a-0273f8a26b48 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:28.644102 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e6cd3d-f199-405c-8f4a-0273f8a26b48 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:28.644365 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e6cd3d-f199-405c-8f4a-0273f8a26b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:28.669319 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-62e6cd3d-f199-405c-8f4a-0273f8a26b48 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:28.671125 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-62e6cd3d-f199-405c-8f4a-0273f8a26b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:28.745949 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1002/2029] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:21:28.754382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e589344-a01f-4370-98ea-ace666d3e90f None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:28.754825 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e589344-a01f-4370-98ea-ace666d3e90f None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:28.755272 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e589344-a01f-4370-98ea-ace666d3e90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:28.787791 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-2e589344-a01f-4370-98ea-ace666d3e90f None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:28.789466 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-2e589344-a01f-4370-98ea-ace666d3e90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:28.818318 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1028/2030] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:28.825585 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e6cd3d-f199-405c-8f4a-0273f8a26b48 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:28.841671 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e6cd3d-f199-405c-8f4a-0273f8a26b48 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:28.849307 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00364e9-fef5-4301-82fc-30c55d715252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:28.851008 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00364e9-fef5-4301-82fc-30c55d715252 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:28.851583 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00364e9-fef5-4301-82fc-30c55d715252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:28.852187 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00364e9-fef5-4301-82fc-30c55d715252 admin admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:28.853215 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00364e9-fef5-4301-82fc-30c55d715252 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:21:28.853533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00364e9-fef5-4301-82fc-30c55d715252 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:28.855443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00364e9-fef5-4301-82fc-30c55d715252 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:28.871864 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1003/2031] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:21:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:21:28.878283 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e589344-a01f-4370-98ea-ace666d3e90f None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:28.890642 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e589344-a01f-4370-98ea-ace666d3e90f None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:28.898333 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739da94-0387-4388-b6f2-794497e366e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:28.904064 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739da94-0387-4388-b6f2-794497e366e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:28.904064 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739da94-0387-4388-b6f2-794497e366e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:28.904064 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739da94-0387-4388-b6f2-794497e366e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:28.904064 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739da94-0387-4388-b6f2-794497e366e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:28.904064 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739da94-0387-4388-b6f2-794497e366e6 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:29.030639 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1029/2032] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:21:28 2022] POST /identity/v3/projects => generated 365 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:21:29.035362 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00364e9-fef5-4301-82fc-30c55d715252 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:29.046356 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00364e9-fef5-4301-82fc-30c55d715252 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:29.123877 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b3f7b3-ca89-4cb9-b638-37caf6a09989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:29.124644 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3f7b3-ca89-4cb9-b638-37caf6a09989 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:29.124936 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3f7b3-ca89-4cb9-b638-37caf6a09989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:29.125215 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3f7b3-ca89-4cb9-b638-37caf6a09989 admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:29.125987 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b3f7b3-ca89-4cb9-b638-37caf6a09989 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:29.128140 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b3f7b3-ca89-4cb9-b638-37caf6a09989 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:29.261175 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1004/2033] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:21:29 2022] POST /identity/v3/users => generated 283 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:21:29.267780 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739da94-0387-4388-b6f2-794497e366e6 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:29.291514 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739da94-0387-4388-b6f2-794497e366e6 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:29.291514 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e546fd7-3b75-4510-9b1e-053f27bb90f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:29.291514 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e546fd7-3b75-4510-9b1e-053f27bb90f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:29.291514 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e546fd7-3b75-4510-9b1e-053f27bb90f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:29.291878 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e546fd7-3b75-4510-9b1e-053f27bb90f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:29.291878 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e546fd7-3b75-4510-9b1e-053f27bb90f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:29.291878 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e546fd7-3b75-4510-9b1e-053f27bb90f2 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:29.303040 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1030/2034] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:21:29.307270 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b3f7b3-ca89-4cb9-b638-37caf6a09989 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:29.318661 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b3f7b3-ca89-4cb9-b638-37caf6a09989 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:29.328168 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec5deb9-b0d6-4452-95e9-5f1d8591df17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:29.328709 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5deb9-b0d6-4452-95e9-5f1d8591df17 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:29.328989 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5deb9-b0d6-4452-95e9-5f1d8591df17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:29.329265 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5deb9-b0d6-4452-95e9-5f1d8591df17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:29.329919 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec5deb9-b0d6-4452-95e9-5f1d8591df17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:29.331539 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec5deb9-b0d6-4452-95e9-5f1d8591df17 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:29.343115 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1005/2035] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:21:29.347324 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e546fd7-3b75-4510-9b1e-053f27bb90f2 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:29.360184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e546fd7-3b75-4510-9b1e-053f27bb90f2 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:29.367629 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f92053-dbad-41ac-aacd-6a38e2b39b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:29.369499 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f92053-dbad-41ac-aacd-6a38e2b39b54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:29.369944 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f92053-dbad-41ac-aacd-6a38e2b39b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:29.371051 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f92053-dbad-41ac-aacd-6a38e2b39b54 admin admin] PATH_INFO: `/v3/projects/76b91e9da2bd454a849168995269fee7/users/b0454173f6984ed69a2ea74c325378be/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:29.393568 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f92053-dbad-41ac-aacd-6a38e2b39b54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76b91e9da2bd454a849168995269fee7, user_id=b0454173f6984ed69a2ea74c325378be, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:29.398767 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f92053-dbad-41ac-aacd-6a38e2b39b54 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:29.501320 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1031/2036] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:21:29 2022] PUT /identity/v3/projects/76b91e9da2bd454a849168995269fee7/users/b0454173f6984ed69a2ea74c325378be/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:29.514281 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d00ec5-3f28-4d76-a7b3-4b9ce3eb57d6 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:29.514697 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d00ec5-3f28-4d76-a7b3-4b9ce3eb57d6 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:29.514974 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d00ec5-3f28-4d76-a7b3-4b9ce3eb57d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:29.550258 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-b2d00ec5-3f28-4d76-a7b3-4b9ce3eb57d6 None None] MFA Rules not processed for user `b0454173f6984ed69a2ea74c325378be`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:29.551693 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-b2d00ec5-3f28-4d76-a7b3-4b9ce3eb57d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:29.634629 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1006/2037] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:29.769232 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f92053-dbad-41ac-aacd-6a38e2b39b54 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:29.771089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-15f92053-dbad-41ac-aacd-6a38e2b39b54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:29.829765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:29.834199 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f92053-dbad-41ac-aacd-6a38e2b39b54 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:29.837863 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08be2b3-53d3-4da5-ba82-6614a0607f92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:29.838472 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08be2b3-53d3-4da5-ba82-6614a0607f92 service cinder] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:29.838758 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08be2b3-53d3-4da5-ba82-6614a0607f92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:29.839034 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08be2b3-53d3-4da5-ba82-6614a0607f92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:29.847599 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08be2b3-53d3-4da5-ba82-6614a0607f92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:29.849030 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08be2b3-53d3-4da5-ba82-6614a0607f92 service cinder] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:29.859316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1032/2038] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:21:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:21:30.071446 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:30.414310 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d00ec5-3f28-4d76-a7b3-4b9ce3eb57d6 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:30.415791 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-b2d00ec5-3f28-4d76-a7b3-4b9ce3eb57d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:30.484891 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d00ec5-3f28-4d76-a7b3-4b9ce3eb57d6 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:30.489542 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53839b82-eff3-4fed-a01d-d1d9c2f98a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:30.490186 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53839b82-eff3-4fed-a01d-d1d9c2f98a75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:30.490472 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53839b82-eff3-4fed-a01d-d1d9c2f98a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:30.490749 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53839b82-eff3-4fed-a01d-d1d9c2f98a75 admin admin] PATH_INFO: `/v3/users/d756e53d759a4535b580cb84a429520d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:30.493138 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53839b82-eff3-4fed-a01d-d1d9c2f98a75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d756e53d759a4535b580cb84a429520d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:30.495068 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53839b82-eff3-4fed-a01d-d1d9c2f98a75 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:30.768802 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-53839b82-eff3-4fed-a01d-d1d9c2f98a75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd756e53d759a4535b580cb84a429520d'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:30.769521 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-53839b82-eff3-4fed-a01d-d1d9c2f98a75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd756e53d759a4535b580cb84a429520d'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:30.770184 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-53839b82-eff3-4fed-a01d-d1d9c2f98a75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd756e53d759a4535b580cb84a429520d'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:30.777365 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-53839b82-eff3-4fed-a01d-d1d9c2f98a75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd756e53d759a4535b580cb84a429520d'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:30.799768 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-53839b82-eff3-4fed-a01d-d1d9c2f98a75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd756e53d759a4535b580cb84a429520d'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:30.877381 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1007/2039] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:21:30 2022] DELETE /identity/v3/users/d756e53d759a4535b580cb84a429520d => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:30.881992 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1eba1-4767-4d99-b539-5f4bf8a4f302 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:30.882496 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1eba1-4767-4d99-b539-5f4bf8a4f302 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:30.882591 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1eba1-4767-4d99-b539-5f4bf8a4f302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:30.907940 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-80c1eba1-4767-4d99-b539-5f4bf8a4f302 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:30.909175 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-80c1eba1-4767-4d99-b539-5f4bf8a4f302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:30.970746 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1033/2040] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:30.980695 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53839b82-eff3-4fed-a01d-d1d9c2f98a75 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:30.982114 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-53839b82-eff3-4fed-a01d-d1d9c2f98a75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:31.041474 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53839b82-eff3-4fed-a01d-d1d9c2f98a75 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:31.045144 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe2903-5bca-42dd-b776-acc0e3b0ac9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:31.045785 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe2903-5bca-42dd-b776-acc0e3b0ac9f service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:31.046067 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe2903-5bca-42dd-b776-acc0e3b0ac9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:31.046348 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe2903-5bca-42dd-b776-acc0e3b0ac9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:31.056122 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe2903-5bca-42dd-b776-acc0e3b0ac9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:31.058109 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe2903-5bca-42dd-b776-acc0e3b0ac9f service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:31.069874 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1008/2041] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:21:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:21:31.329879 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:31.508524 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c1eba1-4767-4d99-b539-5f4bf8a4f302 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:31.509989 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-80c1eba1-4767-4d99-b539-5f4bf8a4f302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:31.524911 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c1eba1-4767-4d99-b539-5f4bf8a4f302 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:31.530111 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072db4de-0278-48b0-a0ed-96f1fb9fae5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:31.530807 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072db4de-0278-48b0-a0ed-96f1fb9fae5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:31.531092 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072db4de-0278-48b0-a0ed-96f1fb9fae5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:31.531373 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072db4de-0278-48b0-a0ed-96f1fb9fae5f admin admin] PATH_INFO: `/v3/projects/76083b06f8174610b8adae0609240077` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:31.532589 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072db4de-0278-48b0-a0ed-96f1fb9fae5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=76083b06f8174610b8adae0609240077)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:31.534653 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072db4de-0278-48b0-a0ed-96f1fb9fae5f admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:31.573672 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe2903-5bca-42dd-b776-acc0e3b0ac9f service neutron] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:31.575257 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-cefe2903-5bca-42dd-b776-acc0e3b0ac9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:31.640591 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe2903-5bca-42dd-b776-acc0e3b0ac9f service neutron] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:31.645812 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c6d03-bf49-4ca9-9b58-a6593d3d8909 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:31.645812 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c6d03-bf49-4ca9-9b58-a6593d3d8909 service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:31.645812 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c6d03-bf49-4ca9-9b58-a6593d3d8909 service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:31.645812 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c6d03-bf49-4ca9-9b58-a6593d3d8909 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:31.647799 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-571c6d03-bf49-4ca9-9b58-a6593d3d8909 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:31.714993 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c6d03-bf49-4ca9-9b58-a6593d3d8909 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:31.715187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-072db4de-0278-48b0-a0ed-96f1fb9fae5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76083b06f8174610b8adae0609240077', 'request_id': 'req-072db4de-0278-48b0-a0ed-96f1fb9fae5f'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:31.716486 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c6d03-bf49-4ca9-9b58-a6593d3d8909 service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:31.722090 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-072db4de-0278-48b0-a0ed-96f1fb9fae5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76083b06f8174610b8adae0609240077', 'request_id': 'req-072db4de-0278-48b0-a0ed-96f1fb9fae5f'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:31.727923 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1009/2042] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:21:31 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:21:31.796908 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-072db4de-0278-48b0-a0ed-96f1fb9fae5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76083b06f8174610b8adae0609240077', 'request_id': 'req-072db4de-0278-48b0-a0ed-96f1fb9fae5f'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:31.812959 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-072db4de-0278-48b0-a0ed-96f1fb9fae5f admin admin] The token cache is being invalidate because project 76083b06f8174610b8adae0609240077 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:21:31.813618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-072db4de-0278-48b0-a0ed-96f1fb9fae5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:31.814402 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-072db4de-0278-48b0-a0ed-96f1fb9fae5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:31.816682 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1034/2043] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:21:31 2022] DELETE /identity/v3/projects/76083b06f8174610b8adae0609240077 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:31.820518 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c6d03-bf49-4ca9-9b58-a6593d3d8909 service cinder] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:31.833468 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c6d03-bf49-4ca9-9b58-a6593d3d8909 service cinder] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:31.909264 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65482c7b-5bad-4e27-b7c3-aade3f6b111a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:31.909913 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65482c7b-5bad-4e27-b7c3-aade3f6b111a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:31.910365 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65482c7b-5bad-4e27-b7c3-aade3f6b111a admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:31.910799 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65482c7b-5bad-4e27-b7c3-aade3f6b111a admin admin] PATH_INFO: `/v3/users/c32582762c9946e29f32c7ff35260aa5` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:31.912166 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65482c7b-5bad-4e27-b7c3-aade3f6b111a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c32582762c9946e29f32c7ff35260aa5)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:31.914605 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65482c7b-5bad-4e27-b7c3-aade3f6b111a admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:31.966479 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:32.115203 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-65482c7b-5bad-4e27-b7c3-aade3f6b111a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c32582762c9946e29f32c7ff35260aa5'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:32.116339 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-65482c7b-5bad-4e27-b7c3-aade3f6b111a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c32582762c9946e29f32c7ff35260aa5'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:32.117299 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-65482c7b-5bad-4e27-b7c3-aade3f6b111a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c32582762c9946e29f32c7ff35260aa5'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:32.126649 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-65482c7b-5bad-4e27-b7c3-aade3f6b111a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c32582762c9946e29f32c7ff35260aa5'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:32.142781 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-65482c7b-5bad-4e27-b7c3-aade3f6b111a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c32582762c9946e29f32c7ff35260aa5'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:32.236935 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1010/2044] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:21:31 2022] DELETE /identity/v3/users/c32582762c9946e29f32c7ff35260aa5 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:21:32.705707 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072db4de-0278-48b0-a0ed-96f1fb9fae5f admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:32.707487 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-072db4de-0278-48b0-a0ed-96f1fb9fae5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:32.721140 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072db4de-0278-48b0-a0ed-96f1fb9fae5f admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:32.790932 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:32.800158 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe713fc-e659-432f-a478-0704143c7140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:32.800806 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe713fc-e659-432f-a478-0704143c7140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:32.801044 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe713fc-e659-432f-a478-0704143c7140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:32.801687 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe713fc-e659-432f-a478-0704143c7140 admin admin] PATH_INFO: `/v3/projects/ca95f9666c62498dbaaea4aeafb4a152` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:32.802937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe713fc-e659-432f-a478-0704143c7140 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca95f9666c62498dbaaea4aeafb4a152)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:32.805458 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe713fc-e659-432f-a478-0704143c7140 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:32.960578 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-9fe713fc-e659-432f-a478-0704143c7140 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca95f9666c62498dbaaea4aeafb4a152', 'request_id': 'req-9fe713fc-e659-432f-a478-0704143c7140'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:32.960578 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-9fe713fc-e659-432f-a478-0704143c7140 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca95f9666c62498dbaaea4aeafb4a152', 'request_id': 'req-9fe713fc-e659-432f-a478-0704143c7140'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:33.356125 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-9fe713fc-e659-432f-a478-0704143c7140 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca95f9666c62498dbaaea4aeafb4a152', 'request_id': 'req-9fe713fc-e659-432f-a478-0704143c7140'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:33.367527 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-9fe713fc-e659-432f-a478-0704143c7140 admin admin] The token cache is being invalidate because project ca95f9666c62498dbaaea4aeafb4a152 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:21:33.367816 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-9fe713fc-e659-432f-a478-0704143c7140 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:33.368496 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-9fe713fc-e659-432f-a478-0704143c7140 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:33.369992 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1035/2045] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:21:32 2022] DELETE /identity/v3/projects/ca95f9666c62498dbaaea4aeafb4a152 => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:33.375713 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ff2b8-907a-49a3-b1c7-036a7e038dc9 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:33.376209 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ff2b8-907a-49a3-b1c7-036a7e038dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:33.376307 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ff2b8-907a-49a3-b1c7-036a7e038dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:33.398710 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-1a6ff2b8-907a-49a3-b1c7-036a7e038dc9 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:33.400179 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-1a6ff2b8-907a-49a3-b1c7-036a7e038dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:33.448244 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:33.463601 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1011/2046] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:33.473082 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72e48b-9d87-4f5a-a48b-c177d7a8d134 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:33.474193 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72e48b-9d87-4f5a-a48b-c177d7a8d134 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:33.474193 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72e48b-9d87-4f5a-a48b-c177d7a8d134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:33.492952 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-1e72e48b-9d87-4f5a-a48b-c177d7a8d134 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:33.497359 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-1e72e48b-9d87-4f5a-a48b-c177d7a8d134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:33.503190 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1036/2047] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:33.507494 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ff2b8-907a-49a3-b1c7-036a7e038dc9 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:33.517809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ff2b8-907a-49a3-b1c7-036a7e038dc9 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:33.522313 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d968fa6-8727-40b9-8428-51b440e3dbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:33.522904 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d968fa6-8727-40b9-8428-51b440e3dbe7 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:33.523182 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d968fa6-8727-40b9-8428-51b440e3dbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:33.523489 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d968fa6-8727-40b9-8428-51b440e3dbe7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:33.524159 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d968fa6-8727-40b9-8428-51b440e3dbe7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:21:33.524478 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d968fa6-8727-40b9-8428-51b440e3dbe7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:33.526036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d968fa6-8727-40b9-8428-51b440e3dbe7 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:33.538329 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1012/2048] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:21:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:21:33.542779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72e48b-9d87-4f5a-a48b-c177d7a8d134 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:33.552189 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72e48b-9d87-4f5a-a48b-c177d7a8d134 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:33.556999 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cce6da-2e32-4d71-a393-a3bf204324c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:33.558361 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cce6da-2e32-4d71-a393-a3bf204324c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:33.558657 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cce6da-2e32-4d71-a393-a3bf204324c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:33.559280 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cce6da-2e32-4d71-a393-a3bf204324c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:33.560026 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cce6da-2e32-4d71-a393-a3bf204324c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:33.562906 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cce6da-2e32-4d71-a393-a3bf204324c1 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:33.735630 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1037/2049] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:21:33 2022] POST /identity/v3/projects => generated 381 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:21:33.739877 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d968fa6-8727-40b9-8428-51b440e3dbe7 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:33.752019 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d968fa6-8727-40b9-8428-51b440e3dbe7 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:33.811785 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8c847-54a6-4b32-91f9-6948cb03b082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:33.812455 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8c847-54a6-4b32-91f9-6948cb03b082 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:33.812736 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8c847-54a6-4b32-91f9-6948cb03b082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:33.813008 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8c847-54a6-4b32-91f9-6948cb03b082 admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:33.813634 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d8c847-54a6-4b32-91f9-6948cb03b082 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:33.815568 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d8c847-54a6-4b32-91f9-6948cb03b082 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:33.928081 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cce6da-2e32-4d71-a393-a3bf204324c1 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:33.929840 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-79cce6da-2e32-4d71-a393-a3bf204324c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:33.939121 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1013/2050] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:21:33 2022] POST /identity/v3/users => generated 291 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:21:33.940383 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8c847-54a6-4b32-91f9-6948cb03b082 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:33.944019 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cce6da-2e32-4d71-a393-a3bf204324c1 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:33.949523 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3e824e-a2b7-45c0-a74b-26e65030a120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:33.950439 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e824e-a2b7-45c0-a74b-26e65030a120 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:33.950832 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8c847-54a6-4b32-91f9-6948cb03b082 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:33.951090 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e824e-a2b7-45c0-a74b-26e65030a120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:33.952315 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e824e-a2b7-45c0-a74b-26e65030a120 admin admin] PATH_INFO: `/v3/users/be651e0d775448d2914ca364904dfd5b` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:33.953503 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3e824e-a2b7-45c0-a74b-26e65030a120 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be651e0d775448d2914ca364904dfd5b)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:33.955534 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3e824e-a2b7-45c0-a74b-26e65030a120 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:33.960250 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be661d-7108-4f36-a6b9-45cd71a4b90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:33.960250 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be661d-7108-4f36-a6b9-45cd71a4b90d admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:33.960250 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be661d-7108-4f36-a6b9-45cd71a4b90d admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:33.960250 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be661d-7108-4f36-a6b9-45cd71a4b90d admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:33.960601 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be661d-7108-4f36-a6b9-45cd71a4b90d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:33.961119 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be661d-7108-4f36-a6b9-45cd71a4b90d admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:33.972557 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1014/2051] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:21:33.976501 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be661d-7108-4f36-a6b9-45cd71a4b90d admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:33.986195 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be661d-7108-4f36-a6b9-45cd71a4b90d admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:33.990913 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec564ebb-3aec-4975-a2cb-30e2aebd5dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:33.991539 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec564ebb-3aec-4975-a2cb-30e2aebd5dea admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:33.991827 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec564ebb-3aec-4975-a2cb-30e2aebd5dea admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:33.992170 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec564ebb-3aec-4975-a2cb-30e2aebd5dea admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:33.992747 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec564ebb-3aec-4975-a2cb-30e2aebd5dea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:33.994336 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec564ebb-3aec-4975-a2cb-30e2aebd5dea admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:34.004402 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1015/2052] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:33 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:21:34.008527 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec564ebb-3aec-4975-a2cb-30e2aebd5dea admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:34.018326 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec564ebb-3aec-4975-a2cb-30e2aebd5dea admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:34.022927 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01b6bdd-447d-448a-bd33-c614ed1db031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:34.023591 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01b6bdd-447d-448a-bd33-c614ed1db031 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:34.023884 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01b6bdd-447d-448a-bd33-c614ed1db031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:34.024163 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01b6bdd-447d-448a-bd33-c614ed1db031 admin admin] PATH_INFO: `/v3/projects/02d169c6132844e287fcbd81c41eaf1e/users/ce28428f480347109d032530e2791127/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:34.039891 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01b6bdd-447d-448a-bd33-c614ed1db031 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02d169c6132844e287fcbd81c41eaf1e, user_id=ce28428f480347109d032530e2791127, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:34.042622 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01b6bdd-447d-448a-bd33-c614ed1db031 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:34.185412 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1016/2053] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:21:34 2022] PUT /identity/v3/projects/02d169c6132844e287fcbd81c41eaf1e/users/ce28428f480347109d032530e2791127/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:21:34.195905 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9399d018-dcd1-4224-ab5f-901d599bc640 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:34.196200 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9399d018-dcd1-4224-ab5f-901d599bc640 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:34.196479 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9399d018-dcd1-4224-ab5f-901d599bc640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:34.223388 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-4a3e824e-a2b7-45c0-a74b-26e65030a120 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be651e0d775448d2914ca364904dfd5b'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:34.226274 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-9399d018-dcd1-4224-ab5f-901d599bc640 None None] MFA Rules not processed for user `ce28428f480347109d032530e2791127`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:34.227546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-9399d018-dcd1-4224-ab5f-901d599bc640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:34.238441 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-4a3e824e-a2b7-45c0-a74b-26e65030a120 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be651e0d775448d2914ca364904dfd5b'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:34.239157 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-4a3e824e-a2b7-45c0-a74b-26e65030a120 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be651e0d775448d2914ca364904dfd5b'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:34.254505 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-4a3e824e-a2b7-45c0-a74b-26e65030a120 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be651e0d775448d2914ca364904dfd5b'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:34.301926 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1017/2054] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:21:34.304808 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9399d018-dcd1-4224-ab5f-901d599bc640 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:34.361871 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-4a3e824e-a2b7-45c0-a74b-26e65030a120 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be651e0d775448d2914ca364904dfd5b'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:34.368452 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1038/2055] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:21:33 2022] DELETE /identity/v3/users/be651e0d775448d2914ca364904dfd5b => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:21:34.374210 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788bcb61-fc34-43cb-bc6a-885c00df6e82 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:34.374508 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788bcb61-fc34-43cb-bc6a-885c00df6e82 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:34.374778 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788bcb61-fc34-43cb-bc6a-885c00df6e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:34.381523 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9399d018-dcd1-4224-ab5f-901d599bc640 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:34.405170 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-788bcb61-fc34-43cb-bc6a-885c00df6e82 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:34.405170 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-788bcb61-fc34-43cb-bc6a-885c00df6e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:34.457615 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7236d-8837-4ce2-87c8-02e8f8b1bdf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:34.458327 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7236d-8837-4ce2-87c8-02e8f8b1bdf7 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:34.458678 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7236d-8837-4ce2-87c8-02e8f8b1bdf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:34.459040 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7236d-8837-4ce2-87c8-02e8f8b1bdf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:34.459787 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c7236d-8837-4ce2-87c8-02e8f8b1bdf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:34.461813 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c7236d-8837-4ce2-87c8-02e8f8b1bdf7 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:34.478202 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1039/2056] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:34.488404 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788bcb61-fc34-43cb-bc6a-885c00df6e82 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:34.489924 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-788bcb61-fc34-43cb-bc6a-885c00df6e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:34.556021 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788bcb61-fc34-43cb-bc6a-885c00df6e82 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:34.559700 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a396bbd-3da2-4d0f-840a-67fe46881e31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:34.560308 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a396bbd-3da2-4d0f-840a-67fe46881e31 service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:34.560616 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a396bbd-3da2-4d0f-840a-67fe46881e31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:34.560887 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a396bbd-3da2-4d0f-840a-67fe46881e31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:34.570489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a396bbd-3da2-4d0f-840a-67fe46881e31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:34.571945 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a396bbd-3da2-4d0f-840a-67fe46881e31 service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:34.581911 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1040/2057] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:21:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:21:34.648337 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1018/2058] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:21:34 2022] POST /identity/v3/projects => generated 379 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:21:34.652590 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a396bbd-3da2-4d0f-840a-67fe46881e31 service neutron] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:34.653998 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-8a396bbd-3da2-4d0f-840a-67fe46881e31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:34.679473 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a396bbd-3da2-4d0f-840a-67fe46881e31 service neutron] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:34.741466 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f144d-aa5f-4193-94e8-b02c76d39002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:34.742087 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f144d-aa5f-4193-94e8-b02c76d39002 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:34.742372 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f144d-aa5f-4193-94e8-b02c76d39002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:34.742649 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f144d-aa5f-4193-94e8-b02c76d39002 admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:34.743286 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f144d-aa5f-4193-94e8-b02c76d39002 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:34.745535 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f144d-aa5f-4193-94e8-b02c76d39002 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:34.836353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1041/2059] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:21:34 2022] POST /identity/v3/users => generated 296 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:21:34.843119 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7236d-8837-4ce2-87c8-02e8f8b1bdf7 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:34.852797 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7236d-8837-4ce2-87c8-02e8f8b1bdf7 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:34.871694 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd247e90-b576-4a78-9293-32777bdfaf8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:34.872371 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd247e90-b576-4a78-9293-32777bdfaf8c admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:34.872634 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd247e90-b576-4a78-9293-32777bdfaf8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:34.872917 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd247e90-b576-4a78-9293-32777bdfaf8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:34.873807 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd247e90-b576-4a78-9293-32777bdfaf8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:34.875385 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd247e90-b576-4a78-9293-32777bdfaf8c admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:34.893434 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1019/2060] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:34 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:21:34.896013 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f144d-aa5f-4193-94e8-b02c76d39002 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:34.915007 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f144d-aa5f-4193-94e8-b02c76d39002 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:34.923342 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d4857-23d7-42d7-aa79-b08514237eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:34.924821 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d4857-23d7-42d7-aa79-b08514237eab admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:34.925308 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d4857-23d7-42d7-aa79-b08514237eab admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:34.925812 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d4857-23d7-42d7-aa79-b08514237eab admin admin] PATH_INFO: `/v3/projects/c62435ed81124c889b0cf3520b04af7e/users/efdf2e943ec245e8a7c9d908b401bfcd/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:34.945217 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0d4857-23d7-42d7-aa79-b08514237eab admin admin] RBAC: Authorizing `identity:create_grant(project_id=c62435ed81124c889b0cf3520b04af7e, user_id=efdf2e943ec245e8a7c9d908b401bfcd, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:34.948631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0d4857-23d7-42d7-aa79-b08514237eab admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:34.990821 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:35.028928 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1042/2061] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:21:34 2022] PUT /identity/v3/projects/c62435ed81124c889b0cf3520b04af7e/users/efdf2e943ec245e8a7c9d908b401bfcd/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:35.033654 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd247e90-b576-4a78-9293-32777bdfaf8c admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:35.043714 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd247e90-b576-4a78-9293-32777bdfaf8c admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:35.100408 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c65b6-05cf-4fea-9c26-34a7e1f95f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:35.100941 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c65b6-05cf-4fea-9c26-34a7e1f95f5e admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:35.101226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c65b6-05cf-4fea-9c26-34a7e1f95f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:35.101510 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c65b6-05cf-4fea-9c26-34a7e1f95f5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:35.102145 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2c65b6-05cf-4fea-9c26-34a7e1f95f5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:35.103773 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2c65b6-05cf-4fea-9c26-34a7e1f95f5e admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:35.113921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1020/2062] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:21:35.121853 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d4857-23d7-42d7-aa79-b08514237eab admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:35.125367 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c65b6-05cf-4fea-9c26-34a7e1f95f5e admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:35.127329 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-6a2c65b6-05cf-4fea-9c26-34a7e1f95f5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:35.131035 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d4857-23d7-42d7-aa79-b08514237eab admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:35.140362 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:35.144369 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c65b6-05cf-4fea-9c26-34a7e1f95f5e admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:35.146392 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809bbea3-6454-4593-9987-5fd031778b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:35.150480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809bbea3-6454-4593-9987-5fd031778b6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:35.150480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809bbea3-6454-4593-9987-5fd031778b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:35.150480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809bbea3-6454-4593-9987-5fd031778b6d admin admin] PATH_INFO: `/v3/projects/c62435ed81124c889b0cf3520b04af7e/users/efdf2e943ec245e8a7c9d908b401bfcd/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:35.151289 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca62b6-4d32-4a8d-9075-8b3b82db5dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:35.151757 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809bbea3-6454-4593-9987-5fd031778b6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c62435ed81124c889b0cf3520b04af7e, user_id=efdf2e943ec245e8a7c9d908b401bfcd, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:35.151935 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca62b6-4d32-4a8d-9075-8b3b82db5dde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:35.151935 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca62b6-4d32-4a8d-9075-8b3b82db5dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:35.151935 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca62b6-4d32-4a8d-9075-8b3b82db5dde admin admin] PATH_INFO: `/v3/projects/f541fae8121b499d93c22d2870afb56f` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:35.152931 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca62b6-4d32-4a8d-9075-8b3b82db5dde admin admin] RBAC: Authorizing `identity:delete_project(project_id=f541fae8121b499d93c22d2870afb56f)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:35.153152 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809bbea3-6454-4593-9987-5fd031778b6d admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:35.155069 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca62b6-4d32-4a8d-9075-8b3b82db5dde admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:35.164152 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1043/2063] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:21:35 2022] PUT /identity/v3/projects/c62435ed81124c889b0cf3520b04af7e/users/efdf2e943ec245e8a7c9d908b401bfcd/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:35.174595 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214456f-f782-4bfc-b0ea-a635faf4cfb6 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:35.174941 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214456f-f782-4bfc-b0ea-a635faf4cfb6 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:35.175223 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214456f-f782-4bfc-b0ea-a635faf4cfb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:35.203916 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-7214456f-f782-4bfc-b0ea-a635faf4cfb6 None None] MFA Rules not processed for user `efdf2e943ec245e8a7c9d908b401bfcd`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:35.205136 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-7214456f-f782-4bfc-b0ea-a635faf4cfb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:35.271810 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1044/2064] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:35 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:21:35.282942 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7214456f-f782-4bfc-b0ea-a635faf4cfb6 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:35.284411 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-7214456f-f782-4bfc-b0ea-a635faf4cfb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:35.291222 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-56ca62b6-4d32-4a8d-9075-8b3b82db5dde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f541fae8121b499d93c22d2870afb56f', 'request_id': 'req-56ca62b6-4d32-4a8d-9075-8b3b82db5dde'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:35.297151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-56ca62b6-4d32-4a8d-9075-8b3b82db5dde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f541fae8121b499d93c22d2870afb56f', 'request_id': 'req-56ca62b6-4d32-4a8d-9075-8b3b82db5dde'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:35.340791 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7214456f-f782-4bfc-b0ea-a635faf4cfb6 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:35.367907 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-56ca62b6-4d32-4a8d-9075-8b3b82db5dde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f541fae8121b499d93c22d2870afb56f', 'request_id': 'req-56ca62b6-4d32-4a8d-9075-8b3b82db5dde'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:35.381865 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-56ca62b6-4d32-4a8d-9075-8b3b82db5dde admin admin] The token cache is being invalidate because project f541fae8121b499d93c22d2870afb56f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:21:35.382536 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-56ca62b6-4d32-4a8d-9075-8b3b82db5dde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:35.382892 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-56ca62b6-4d32-4a8d-9075-8b3b82db5dde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:35.384443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1021/2065] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:21:35 2022] DELETE /identity/v3/projects/f541fae8121b499d93c22d2870afb56f => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:21:35.388523 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca62b6-4d32-4a8d-9075-8b3b82db5dde admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:35.391224 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-56ca62b6-4d32-4a8d-9075-8b3b82db5dde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:35.393712 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc3d636-4134-479e-bc84-01fc19b7ffca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:35.394338 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3d636-4134-479e-bc84-01fc19b7ffca service cinder] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:35.394630 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3d636-4134-479e-bc84-01fc19b7ffca service cinder] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:35.394974 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3d636-4134-479e-bc84-01fc19b7ffca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:35.396780 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-8cc3d636-4134-479e-bc84-01fc19b7ffca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:35.456620 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca62b6-4d32-4a8d-9075-8b3b82db5dde admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:35.462949 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa23c9-9650-4486-8cb7-e5deaac648d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:35.463634 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa23c9-9650-4486-8cb7-e5deaac648d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:35.463918 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa23c9-9650-4486-8cb7-e5deaac648d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:35.464193 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa23c9-9650-4486-8cb7-e5deaac648d5 admin admin] PATH_INFO: `/v3/users/152c1070e5964c469d2790f2cfe85236` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:35.465466 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa23c9-9650-4486-8cb7-e5deaac648d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=152c1070e5964c469d2790f2cfe85236)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:35.467514 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa23c9-9650-4486-8cb7-e5deaac648d5 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:35.478437 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc3d636-4134-479e-bc84-01fc19b7ffca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:35.478437 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc3d636-4134-479e-bc84-01fc19b7ffca service cinder] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:35.485114 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1045/2066] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:21:35 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:21:35.758187 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-07aa23c9-9650-4486-8cb7-e5deaac648d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '152c1070e5964c469d2790f2cfe85236'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:35.759008 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-07aa23c9-9650-4486-8cb7-e5deaac648d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '152c1070e5964c469d2790f2cfe85236'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:35.759736 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-07aa23c9-9650-4486-8cb7-e5deaac648d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '152c1070e5964c469d2790f2cfe85236'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:35.783944 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-07aa23c9-9650-4486-8cb7-e5deaac648d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '152c1070e5964c469d2790f2cfe85236'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:35.783944 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-07aa23c9-9650-4486-8cb7-e5deaac648d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '152c1070e5964c469d2790f2cfe85236'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:35.830916 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:35.865838 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1022/2067] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:21:35 2022] DELETE /identity/v3/users/152c1070e5964c469d2790f2cfe85236 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:21:35.929341 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:36.084594 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc3d636-4134-479e-bc84-01fc19b7ffca service cinder] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:36.086068 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-8cc3d636-4134-479e-bc84-01fc19b7ffca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:36.100230 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc3d636-4134-479e-bc84-01fc19b7ffca service cinder] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:36.162807 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f50757-80b2-46d6-bcf8-164b76ed28a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:36.163361 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f50757-80b2-46d6-bcf8-164b76ed28a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:36.163672 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f50757-80b2-46d6-bcf8-164b76ed28a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:36.163983 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f50757-80b2-46d6-bcf8-164b76ed28a6 admin admin] PATH_INFO: `/v3/users/ce28428f480347109d032530e2791127` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:36.165159 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f50757-80b2-46d6-bcf8-164b76ed28a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce28428f480347109d032530e2791127)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:36.167132 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f50757-80b2-46d6-bcf8-164b76ed28a6 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:36.288589 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa23c9-9650-4486-8cb7-e5deaac648d5 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:36.292419 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-07aa23c9-9650-4486-8cb7-e5deaac648d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:36.307729 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa23c9-9650-4486-8cb7-e5deaac648d5 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:36.313129 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a0dc9-9d95-4469-aa1f-7b8972dfce85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:36.313802 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a0dc9-9d95-4469-aa1f-7b8972dfce85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:36.314438 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a0dc9-9d95-4469-aa1f-7b8972dfce85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:36.314596 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a0dc9-9d95-4469-aa1f-7b8972dfce85 admin admin] PATH_INFO: `/v3/projects/21fbb46ad3a949da935fd35b22bcc60f` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:36.315889 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a0dc9-9d95-4469-aa1f-7b8972dfce85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21fbb46ad3a949da935fd35b22bcc60f)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:36.317871 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a0dc9-9d95-4469-aa1f-7b8972dfce85 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:36.356946 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:36.391001 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-f3f50757-80b2-46d6-bcf8-164b76ed28a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce28428f480347109d032530e2791127'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:36.399841 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-f3f50757-80b2-46d6-bcf8-164b76ed28a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce28428f480347109d032530e2791127'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:36.400848 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-f3f50757-80b2-46d6-bcf8-164b76ed28a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce28428f480347109d032530e2791127'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:36.414708 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-f3f50757-80b2-46d6-bcf8-164b76ed28a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce28428f480347109d032530e2791127'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:36.416834 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-4b7a0dc9-9d95-4469-aa1f-7b8972dfce85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21fbb46ad3a949da935fd35b22bcc60f', 'request_id': 'req-4b7a0dc9-9d95-4469-aa1f-7b8972dfce85'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:36.423156 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-4b7a0dc9-9d95-4469-aa1f-7b8972dfce85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21fbb46ad3a949da935fd35b22bcc60f', 'request_id': 'req-4b7a0dc9-9d95-4469-aa1f-7b8972dfce85'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:36.426897 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:36.459102 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-f3f50757-80b2-46d6-bcf8-164b76ed28a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce28428f480347109d032530e2791127'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:36.465420 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1046/2068] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:21:36 2022] DELETE /identity/v3/users/ce28428f480347109d032530e2791127 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:36.470532 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbe20d-f2ee-4422-9bc6-da936abb8622 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:36.470532 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbe20d-f2ee-4422-9bc6-da936abb8622 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:36.470785 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbe20d-f2ee-4422-9bc6-da936abb8622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:36.489131 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-e9bbe20d-f2ee-4422-9bc6-da936abb8622 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:36.490339 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-e9bbe20d-f2ee-4422-9bc6-da936abb8622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:36.525432 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-4b7a0dc9-9d95-4469-aa1f-7b8972dfce85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21fbb46ad3a949da935fd35b22bcc60f', 'request_id': 'req-4b7a0dc9-9d95-4469-aa1f-7b8972dfce85'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:36.537662 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-4b7a0dc9-9d95-4469-aa1f-7b8972dfce85 admin admin] The token cache is being invalidate because project 21fbb46ad3a949da935fd35b22bcc60f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:21:36.537955 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-4b7a0dc9-9d95-4469-aa1f-7b8972dfce85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:36.538617 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-4b7a0dc9-9d95-4469-aa1f-7b8972dfce85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:36.540179 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1023/2069] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:21:36 2022] DELETE /identity/v3/projects/21fbb46ad3a949da935fd35b22bcc60f => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:21:36.544968 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d7dde-4839-4780-95b0-2eff41d36a47 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:36.545338 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d7dde-4839-4780-95b0-2eff41d36a47 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:36.545621 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d7dde-4839-4780-95b0-2eff41d36a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:36.551838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1047/2070] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:21:36.562287 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bbe20d-f2ee-4422-9bc6-da936abb8622 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:36.563765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-e9bbe20d-f2ee-4422-9bc6-da936abb8622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:36.565763 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-141d7dde-4839-4780-95b0-2eff41d36a47 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:36.566955 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-141d7dde-4839-4780-95b0-2eff41d36a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:36.575484 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1024/2071] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:21:36.588007 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f82f8-698b-45a1-9a6a-b820d70bf4f4 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:36.588288 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f82f8-698b-45a1-9a6a-b820d70bf4f4 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:36.588569 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f82f8-698b-45a1-9a6a-b820d70bf4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:36.605680 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:36.610082 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-b38f82f8-698b-45a1-9a6a-b820d70bf4f4 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:36.610899 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-b38f82f8-698b-45a1-9a6a-b820d70bf4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:36.619169 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1025/2072] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:21:36.624389 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f82f8-698b-45a1-9a6a-b820d70bf4f4 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:36.624915 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bbe20d-f2ee-4422-9bc6-da936abb8622 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:36.629438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d098591-0cee-4b07-ad58-7b56e7ab2e6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:36.633193 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d098591-0cee-4b07-ad58-7b56e7ab2e6e service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:36.633193 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d098591-0cee-4b07-ad58-7b56e7ab2e6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:36.633193 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d098591-0cee-4b07-ad58-7b56e7ab2e6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:36.633606 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f82f8-698b-45a1-9a6a-b820d70bf4f4 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:36.638345 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956cad13-0624-421f-be13-0d18a9641392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:36.638993 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956cad13-0624-421f-be13-0d18a9641392 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:36.639863 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956cad13-0624-421f-be13-0d18a9641392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:36.640137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956cad13-0624-421f-be13-0d18a9641392 admin admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:36.640794 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956cad13-0624-421f-be13-0d18a9641392 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:21:36.641149 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956cad13-0624-421f-be13-0d18a9641392 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:36.641372 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d098591-0cee-4b07-ad58-7b56e7ab2e6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:36.642686 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956cad13-0624-421f-be13-0d18a9641392 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:36.642794 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d098591-0cee-4b07-ad58-7b56e7ab2e6e service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:36.652485 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1048/2073] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:21:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:21:36.655343 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1026/2074] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:21:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:21:36.664714 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d098591-0cee-4b07-ad58-7b56e7ab2e6e service neutron] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:36.674436 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d098591-0cee-4b07-ad58-7b56e7ab2e6e service neutron] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:36.680074 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a104105-caa4-4314-a13b-887adb81d4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:36.680965 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a104105-caa4-4314-a13b-887adb81d4ee admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:36.681448 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a104105-caa4-4314-a13b-887adb81d4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:36.682054 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a104105-caa4-4314-a13b-887adb81d4ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:36.682739 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a104105-caa4-4314-a13b-887adb81d4ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:36.684781 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a104105-caa4-4314-a13b-887adb81d4ee admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:36.795230 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1049/2075] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:21:36 2022] POST /identity/v3/projects => generated 381 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:21:36.798128 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956cad13-0624-421f-be13-0d18a9641392 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:36.809826 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956cad13-0624-421f-be13-0d18a9641392 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:36.881800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07592c06-fbf6-4c14-80ea-e8d025a2d6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:36.882785 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07592c06-fbf6-4c14-80ea-e8d025a2d6cf admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:36.883271 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07592c06-fbf6-4c14-80ea-e8d025a2d6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:36.883764 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07592c06-fbf6-4c14-80ea-e8d025a2d6cf admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:36.884587 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07592c06-fbf6-4c14-80ea-e8d025a2d6cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:36.886612 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07592c06-fbf6-4c14-80ea-e8d025a2d6cf admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:37.017841 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1027/2076] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:21:36 2022] POST /identity/v3/users => generated 291 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:21:37.022906 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a104105-caa4-4314-a13b-887adb81d4ee admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:37.037957 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a104105-caa4-4314-a13b-887adb81d4ee admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:37.043597 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80088c9-e35a-4dc0-b8d6-dd84e531f8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:37.044211 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80088c9-e35a-4dc0-b8d6-dd84e531f8c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:37.044489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80088c9-e35a-4dc0-b8d6-dd84e531f8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:37.044758 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80088c9-e35a-4dc0-b8d6-dd84e531f8c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:37.045435 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80088c9-e35a-4dc0-b8d6-dd84e531f8c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:37.047018 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80088c9-e35a-4dc0-b8d6-dd84e531f8c8 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:37.059025 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1050/2077] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:21:37.065466 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07592c06-fbf6-4c14-80ea-e8d025a2d6cf admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:37.078846 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07592c06-fbf6-4c14-80ea-e8d025a2d6cf admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:37.083863 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf9e9d-5d0d-4270-9c1b-63f0eac70536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:37.084466 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf9e9d-5d0d-4270-9c1b-63f0eac70536 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:37.084753 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf9e9d-5d0d-4270-9c1b-63f0eac70536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:37.085031 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf9e9d-5d0d-4270-9c1b-63f0eac70536 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:37.085675 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bf9e9d-5d0d-4270-9c1b-63f0eac70536 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:37.088133 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bf9e9d-5d0d-4270-9c1b-63f0eac70536 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:37.100121 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1028/2078] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:21:37.104356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80088c9-e35a-4dc0-b8d6-dd84e531f8c8 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:37.117648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80088c9-e35a-4dc0-b8d6-dd84e531f8c8 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:37.134484 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e800b-5f7d-489b-9ec0-6b34e4301b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:37.135279 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e800b-5f7d-489b-9ec0-6b34e4301b4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:37.135618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e800b-5f7d-489b-9ec0-6b34e4301b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:37.135903 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e800b-5f7d-489b-9ec0-6b34e4301b4c admin admin] PATH_INFO: `/v3/projects/f0011eaee5b24e8aaa4e7b08263fbe04/users/67d48cb065e9467e91a914742513fb01/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:37.153559 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e800b-5f7d-489b-9ec0-6b34e4301b4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0011eaee5b24e8aaa4e7b08263fbe04, user_id=67d48cb065e9467e91a914742513fb01, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:37.157091 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e800b-5f7d-489b-9ec0-6b34e4301b4c admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:37.185640 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf9e9d-5d0d-4270-9c1b-63f0eac70536 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:37.185640 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-22bf9e9d-5d0d-4270-9c1b-63f0eac70536 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:37.197021 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf9e9d-5d0d-4270-9c1b-63f0eac70536 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:37.202662 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ae143c-1645-4096-a2b3-8424f11d0508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:37.203299 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ae143c-1645-4096-a2b3-8424f11d0508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:37.203638 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ae143c-1645-4096-a2b3-8424f11d0508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:37.203913 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ae143c-1645-4096-a2b3-8424f11d0508 admin admin] PATH_INFO: `/v3/projects/02d169c6132844e287fcbd81c41eaf1e` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:37.205588 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ae143c-1645-4096-a2b3-8424f11d0508 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02d169c6132844e287fcbd81c41eaf1e)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:37.207606 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ae143c-1645-4096-a2b3-8424f11d0508 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:37.281054 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1051/2079] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:21:37 2022] PUT /identity/v3/projects/f0011eaee5b24e8aaa4e7b08263fbe04/users/67d48cb065e9467e91a914742513fb01/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:21:37.291617 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a91011-e9ec-4afe-81ba-9018a8c39dff None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:37.291936 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a91011-e9ec-4afe-81ba-9018a8c39dff None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:37.292219 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a91011-e9ec-4afe-81ba-9018a8c39dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:37.328797 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-81ae143c-1645-4096-a2b3-8424f11d0508 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02d169c6132844e287fcbd81c41eaf1e', 'request_id': 'req-81ae143c-1645-4096-a2b3-8424f11d0508'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:37.336572 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-81ae143c-1645-4096-a2b3-8424f11d0508 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02d169c6132844e287fcbd81c41eaf1e', 'request_id': 'req-81ae143c-1645-4096-a2b3-8424f11d0508'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:37.339337 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:DELETE=12,keystone:SELECT=83,keystone:INSERT=5 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:37.343518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-a0a91011-e9ec-4afe-81ba-9018a8c39dff None None] MFA Rules not processed for user `67d48cb065e9467e91a914742513fb01`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:37.344777 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-a0a91011-e9ec-4afe-81ba-9018a8c39dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:37.413311 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1052/2080] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:21:37.421805 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a91011-e9ec-4afe-81ba-9018a8c39dff None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:37.421952 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-81ae143c-1645-4096-a2b3-8424f11d0508 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02d169c6132844e287fcbd81c41eaf1e', 'request_id': 'req-81ae143c-1645-4096-a2b3-8424f11d0508'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:37.431517 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-81ae143c-1645-4096-a2b3-8424f11d0508 admin admin] The token cache is being invalidate because project 02d169c6132844e287fcbd81c41eaf1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:21:37.431836 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-81ae143c-1645-4096-a2b3-8424f11d0508 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:37.432494 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-81ae143c-1645-4096-a2b3-8424f11d0508 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:37.434183 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1029/2081] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:21:37 2022] DELETE /identity/v3/projects/02d169c6132844e287fcbd81c41eaf1e => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:21:37.438638 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ae143c-1645-4096-a2b3-8424f11d0508 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:37.441420 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-81ae143c-1645-4096-a2b3-8424f11d0508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:37.489026 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a91011-e9ec-4afe-81ba-9018a8c39dff None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:37.493850 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c95316-69da-45f3-8f52-a7e87ba5e286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:37.494464 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c95316-69da-45f3-8f52-a7e87ba5e286 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:37.494772 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c95316-69da-45f3-8f52-a7e87ba5e286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:37.495447 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c95316-69da-45f3-8f52-a7e87ba5e286 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:37.496104 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c95316-69da-45f3-8f52-a7e87ba5e286 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:37.500074 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c95316-69da-45f3-8f52-a7e87ba5e286 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:37.509369 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ae143c-1645-4096-a2b3-8424f11d0508 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:37.515929 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe17ab-89c4-42fc-9e72-bee69d301b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:37.521317 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe17ab-89c4-42fc-9e72-bee69d301b9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:37.521317 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe17ab-89c4-42fc-9e72-bee69d301b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:37.521317 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe17ab-89c4-42fc-9e72-bee69d301b9d admin admin] PATH_INFO: `/v3/users/efdf2e943ec245e8a7c9d908b401bfcd` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:37.521317 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe17ab-89c4-42fc-9e72-bee69d301b9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=efdf2e943ec245e8a7c9d908b401bfcd)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:37.521317 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe17ab-89c4-42fc-9e72-bee69d301b9d admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:37.521737 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:37.615509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1053/2082] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:21:37 2022] POST /identity/v3/projects => generated 379 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:21:37.626965 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c95316-69da-45f3-8f52-a7e87ba5e286 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:37.628603 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-b4c95316-69da-45f3-8f52-a7e87ba5e286 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:37.699372 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c95316-69da-45f3-8f52-a7e87ba5e286 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:37.706213 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b4693-d9ce-4355-8c6c-7b08e841fd65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:37.709469 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b4693-d9ce-4355-8c6c-7b08e841fd65 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:37.709469 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b4693-d9ce-4355-8c6c-7b08e841fd65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:37.709469 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b4693-d9ce-4355-8c6c-7b08e841fd65 admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:37.709469 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611b4693-d9ce-4355-8c6c-7b08e841fd65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:37.710147 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611b4693-d9ce-4355-8c6c-7b08e841fd65 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:37.741468 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-4afe17ab-89c4-42fc-9e72-bee69d301b9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efdf2e943ec245e8a7c9d908b401bfcd'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:37.742127 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-4afe17ab-89c4-42fc-9e72-bee69d301b9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efdf2e943ec245e8a7c9d908b401bfcd'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:37.742753 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-4afe17ab-89c4-42fc-9e72-bee69d301b9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efdf2e943ec245e8a7c9d908b401bfcd'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:37.749993 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-4afe17ab-89c4-42fc-9e72-bee69d301b9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efdf2e943ec245e8a7c9d908b401bfcd'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:37.761576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-4afe17ab-89c4-42fc-9e72-bee69d301b9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efdf2e943ec245e8a7c9d908b401bfcd'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:37.843476 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1054/2083] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:21:37 2022] POST /identity/v3/users => generated 296 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:21:37.847656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b4693-d9ce-4355-8c6c-7b08e841fd65 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:37.849378 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-611b4693-d9ce-4355-8c6c-7b08e841fd65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:37.862251 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b4693-d9ce-4355-8c6c-7b08e841fd65 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:37.867610 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a89ebf-c581-4f8b-af9c-72f32bf8ffdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:37.868479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a89ebf-c581-4f8b-af9c-72f32bf8ffdb admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:37.868785 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a89ebf-c581-4f8b-af9c-72f32bf8ffdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:37.869077 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a89ebf-c581-4f8b-af9c-72f32bf8ffdb admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:37.876076 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a89ebf-c581-4f8b-af9c-72f32bf8ffdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:37.876246 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1030/2084] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:21:37 2022] DELETE /identity/v3/users/efdf2e943ec245e8a7c9d908b401bfcd => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:21:37.876363 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a89ebf-c581-4f8b-af9c-72f32bf8ffdb admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:37.890315 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1055/2085] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 12:21:37.891597 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe17ab-89c4-42fc-9e72-bee69d301b9d admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:38.010313 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe17ab-89c4-42fc-9e72-bee69d301b9d admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:38.017617 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbc8e6-9408-475f-b783-f2ad575448e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:38.018338 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbc8e6-9408-475f-b783-f2ad575448e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:38.023615 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbc8e6-9408-475f-b783-f2ad575448e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:38.023615 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbc8e6-9408-475f-b783-f2ad575448e8 admin admin] PATH_INFO: `/v3/projects/d896595976a448ffba8b3faa25d130a1/users/eeab41f9661b4c97af4aef608eaca045/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:38.049564 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:38.056362 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfbc8e6-9408-475f-b783-f2ad575448e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d896595976a448ffba8b3faa25d130a1, user_id=eeab41f9661b4c97af4aef608eaca045, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:38.061857 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfbc8e6-9408-475f-b783-f2ad575448e8 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:38.225351 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1031/2086] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:21:37 2022] PUT /identity/v3/projects/d896595976a448ffba8b3faa25d130a1/users/eeab41f9661b4c97af4aef608eaca045/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:38.226566 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a89ebf-c581-4f8b-af9c-72f32bf8ffdb admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:38.237222 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a89ebf-c581-4f8b-af9c-72f32bf8ffdb admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:38.312547 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af902a-7d1b-42af-8ffc-9b1f1311a03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:38.313284 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af902a-7d1b-42af-8ffc-9b1f1311a03f admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:38.313641 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af902a-7d1b-42af-8ffc-9b1f1311a03f admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:38.313993 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af902a-7d1b-42af-8ffc-9b1f1311a03f admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:38.314822 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12af902a-7d1b-42af-8ffc-9b1f1311a03f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:38.316825 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12af902a-7d1b-42af-8ffc-9b1f1311a03f admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:38.328044 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1056/2087] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:21:38.332182 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbc8e6-9408-475f-b783-f2ad575448e8 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:38.343585 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbc8e6-9408-475f-b783-f2ad575448e8 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:38.348469 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7293fc-c2b9-4e79-b5e4-ae5fcc3c9ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:38.349266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7293fc-c2b9-4e79-b5e4-ae5fcc3c9ff6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:38.349556 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7293fc-c2b9-4e79-b5e4-ae5fcc3c9ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:38.349835 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7293fc-c2b9-4e79-b5e4-ae5fcc3c9ff6 admin admin] PATH_INFO: `/v3/projects/d896595976a448ffba8b3faa25d130a1/users/eeab41f9661b4c97af4aef608eaca045/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:38.351997 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7293fc-c2b9-4e79-b5e4-ae5fcc3c9ff6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d896595976a448ffba8b3faa25d130a1, user_id=eeab41f9661b4c97af4aef608eaca045, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:38.354661 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7293fc-c2b9-4e79-b5e4-ae5fcc3c9ff6 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:38.366864 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1032/2088] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:21:38 2022] PUT /identity/v3/projects/d896595976a448ffba8b3faa25d130a1/users/eeab41f9661b4c97af4aef608eaca045/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:21:38.378414 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6aa979-ac72-4ace-bda4-ef1fd00e73fa None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:38.378706 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6aa979-ac72-4ace-bda4-ef1fd00e73fa None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:38.378985 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6aa979-ac72-4ace-bda4-ef1fd00e73fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:38.409254 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-1d6aa979-ac72-4ace-bda4-ef1fd00e73fa None None] MFA Rules not processed for user `eeab41f9661b4c97af4aef608eaca045`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:38.410501 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-1d6aa979-ac72-4ace-bda4-ef1fd00e73fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:38.472110 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:38.481308 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1057/2089] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:38 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:21:38.497322 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7293fc-c2b9-4e79-b5e4-ae5fcc3c9ff6 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:38.499231 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-fc7293fc-c2b9-4e79-b5e4-ae5fcc3c9ff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:38.575055 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7293fc-c2b9-4e79-b5e4-ae5fcc3c9ff6 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:38.580466 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad236689-9510-48c1-aacd-391316b4d265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:38.581126 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad236689-9510-48c1-aacd-391316b4d265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:38.581406 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad236689-9510-48c1-aacd-391316b4d265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:38.581692 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad236689-9510-48c1-aacd-391316b4d265 admin admin] PATH_INFO: `/v3/projects/c62435ed81124c889b0cf3520b04af7e` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:38.582800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad236689-9510-48c1-aacd-391316b4d265 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c62435ed81124c889b0cf3520b04af7e)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:38.584827 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad236689-9510-48c1-aacd-391316b4d265 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:38.729327 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-ad236689-9510-48c1-aacd-391316b4d265 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c62435ed81124c889b0cf3520b04af7e', 'request_id': 'req-ad236689-9510-48c1-aacd-391316b4d265'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:38.743707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-ad236689-9510-48c1-aacd-391316b4d265 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c62435ed81124c889b0cf3520b04af7e', 'request_id': 'req-ad236689-9510-48c1-aacd-391316b4d265'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:38.803163 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6aa979-ac72-4ace-bda4-ef1fd00e73fa None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:38.804612 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-1d6aa979-ac72-4ace-bda4-ef1fd00e73fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:38.811254 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-ad236689-9510-48c1-aacd-391316b4d265 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c62435ed81124c889b0cf3520b04af7e', 'request_id': 'req-ad236689-9510-48c1-aacd-391316b4d265'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:38.830520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-ad236689-9510-48c1-aacd-391316b4d265 admin admin] The token cache is being invalidate because project c62435ed81124c889b0cf3520b04af7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:21:38.831325 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-ad236689-9510-48c1-aacd-391316b4d265 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:38.832609 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-ad236689-9510-48c1-aacd-391316b4d265 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:38.840582 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1033/2090] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:21:38 2022] DELETE /identity/v3/projects/c62435ed81124c889b0cf3520b04af7e => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:38.843910 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb7b75-e760-4e37-afd3-0e1b8daaa812 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:38.844200 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb7b75-e760-4e37-afd3-0e1b8daaa812 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:38.844478 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb7b75-e760-4e37-afd3-0e1b8daaa812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:38.873196 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-73eb7b75-e760-4e37-afd3-0e1b8daaa812 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:38.874543 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-73eb7b75-e760-4e37-afd3-0e1b8daaa812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:38.906087 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6aa979-ac72-4ace-bda4-ef1fd00e73fa None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:38.951737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1034/2091] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:38.960382 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2b9fa-232e-4f2a-8a16-7c90c409f167 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:38.960893 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2b9fa-232e-4f2a-8a16-7c90c409f167 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:38.961234 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2b9fa-232e-4f2a-8a16-7c90c409f167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:38.968267 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d038a1-caff-4181-ac14-5b70f724a23f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:38.968886 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d038a1-caff-4181-ac14-5b70f724a23f service cinder] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:38.975995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d038a1-caff-4181-ac14-5b70f724a23f service cinder] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:38.975995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d038a1-caff-4181-ac14-5b70f724a23f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:38.975995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-22d038a1-caff-4181-ac14-5b70f724a23f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:38.981826 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-4ea2b9fa-232e-4f2a-8a16-7c90c409f167 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:38.983663 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-4ea2b9fa-232e-4f2a-8a16-7c90c409f167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:38.995877 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1035/2092] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:38.997268 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea2b9fa-232e-4f2a-8a16-7c90c409f167 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:39.001681 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:39.007353 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea2b9fa-232e-4f2a-8a16-7c90c409f167 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:39.012067 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e556605-56ea-47c5-98df-5025d66237d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:39.012659 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e556605-56ea-47c5-98df-5025d66237d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:39.012937 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e556605-56ea-47c5-98df-5025d66237d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:39.013208 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e556605-56ea-47c5-98df-5025d66237d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:39.013943 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e556605-56ea-47c5-98df-5025d66237d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:21:39.014378 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e556605-56ea-47c5-98df-5025d66237d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:39.016001 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e556605-56ea-47c5-98df-5025d66237d1 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:39.036499 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1036/2093] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:21:39.037576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e556605-56ea-47c5-98df-5025d66237d1 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:39.049250 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d038a1-caff-4181-ac14-5b70f724a23f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:39.049735 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e556605-56ea-47c5-98df-5025d66237d1 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:39.051018 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d038a1-caff-4181-ac14-5b70f724a23f service cinder] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:39.054959 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e1639-81e1-4dfc-93b5-0630144e7fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:39.055467 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e1639-81e1-4dfc-93b5-0630144e7fe5 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:39.055768 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e1639-81e1-4dfc-93b5-0630144e7fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:39.056037 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e1639-81e1-4dfc-93b5-0630144e7fe5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:39.056653 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e1639-81e1-4dfc-93b5-0630144e7fe5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:39.058592 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e1639-81e1-4dfc-93b5-0630144e7fe5 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:39.061163 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1058/2094] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:21:38 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:21:39.192837 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1037/2095] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:21:39 2022] POST /identity/v3/projects => generated 373 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:21:39.194306 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d038a1-caff-4181-ac14-5b70f724a23f service cinder] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:39.222626 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d038a1-caff-4181-ac14-5b70f724a23f service cinder] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:39.313845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006020b-d3b0-48e8-9eb9-c09f3c567e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:39.314554 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006020b-d3b0-48e8-9eb9-c09f3c567e58 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:39.314757 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006020b-d3b0-48e8-9eb9-c09f3c567e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:39.315023 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006020b-d3b0-48e8-9eb9-c09f3c567e58 admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:39.315695 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4006020b-d3b0-48e8-9eb9-c09f3c567e58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:39.317804 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4006020b-d3b0-48e8-9eb9-c09f3c567e58 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:39.328878 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:39.445820 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1059/2096] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:21:39 2022] POST /identity/v3/users => generated 287 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:21:39.449950 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e1639-81e1-4dfc-93b5-0630144e7fe5 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:39.460772 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e1639-81e1-4dfc-93b5-0630144e7fe5 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:39.465573 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf918df-dba5-4b60-81a7-65f7afdd524b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:39.466193 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf918df-dba5-4b60-81a7-65f7afdd524b admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:39.466551 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf918df-dba5-4b60-81a7-65f7afdd524b admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:39.466972 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf918df-dba5-4b60-81a7-65f7afdd524b admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:39.467518 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf918df-dba5-4b60-81a7-65f7afdd524b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:39.469990 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf918df-dba5-4b60-81a7-65f7afdd524b admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:39.488353 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1038/2097] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:21:39.489774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006020b-d3b0-48e8-9eb9-c09f3c567e58 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:39.500954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006020b-d3b0-48e8-9eb9-c09f3c567e58 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:39.506367 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d3694-a0b8-44ca-a055-a8a1c080942e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:39.507178 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d3694-a0b8-44ca-a055-a8a1c080942e admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:39.507622 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d3694-a0b8-44ca-a055-a8a1c080942e admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:39.507990 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d3694-a0b8-44ca-a055-a8a1c080942e admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:39.508995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d3694-a0b8-44ca-a055-a8a1c080942e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:39.510688 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d3694-a0b8-44ca-a055-a8a1c080942e admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:39.521426 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1060/2098] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:21:39.531545 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf918df-dba5-4b60-81a7-65f7afdd524b admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:39.543284 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf918df-dba5-4b60-81a7-65f7afdd524b admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:39.548413 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc593cb-8591-499d-a014-cccb23e36fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:39.549076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc593cb-8591-499d-a014-cccb23e36fcb admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:39.549365 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc593cb-8591-499d-a014-cccb23e36fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:39.549645 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc593cb-8591-499d-a014-cccb23e36fcb admin admin] PATH_INFO: `/v3/projects/9b2edbbd46f54ad29ce12dad1762aaa7/users/991cdaf69fd5439c9cad12cc2be49ada/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:39.565773 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc593cb-8591-499d-a014-cccb23e36fcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b2edbbd46f54ad29ce12dad1762aaa7, user_id=991cdaf69fd5439c9cad12cc2be49ada, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:39.568527 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc593cb-8591-499d-a014-cccb23e36fcb admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:39.613682 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d3694-a0b8-44ca-a055-a8a1c080942e admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:39.615747 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-393d3694-a0b8-44ca-a055-a8a1c080942e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:39.680064 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d3694-a0b8-44ca-a055-a8a1c080942e admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:39.683922 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2cf0b2-55e7-4adb-9078-01b4fd0cae7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:39.684529 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2cf0b2-55e7-4adb-9078-01b4fd0cae7d service cinder] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:39.684817 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2cf0b2-55e7-4adb-9078-01b4fd0cae7d service cinder] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:39.685090 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2cf0b2-55e7-4adb-9078-01b4fd0cae7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:39.687767 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-3f2cf0b2-55e7-4adb-9078-01b4fd0cae7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:39.695027 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1039/2099] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:21:39 2022] PUT /identity/v3/projects/9b2edbbd46f54ad29ce12dad1762aaa7/users/991cdaf69fd5439c9cad12cc2be49ada/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:39.699304 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc593cb-8591-499d-a014-cccb23e36fcb admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:39.699304 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-fbc593cb-8591-499d-a014-cccb23e36fcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:39.759481 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2cf0b2-55e7-4adb-9078-01b4fd0cae7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:39.760830 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2cf0b2-55e7-4adb-9078-01b4fd0cae7d service cinder] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:39.765051 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc593cb-8591-499d-a014-cccb23e36fcb admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:39.770231 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fae43f-4eac-48ac-b6a0-b39f833065f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:39.770640 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1061/2100] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:21:39 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:21:39.771005 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fae43f-4eac-48ac-b6a0-b39f833065f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:39.771787 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fae43f-4eac-48ac-b6a0-b39f833065f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:39.771946 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a0bf6-d8a0-49be-b998-df590094785e None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:39.772163 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a0bf6-d8a0-49be-b998-df590094785e None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:39.772300 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fae43f-4eac-48ac-b6a0-b39f833065f3 admin admin] PATH_INFO: `/v3/users/ac825664b4324c8da293b2aa9c3a9f9e` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:39.772408 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a0bf6-d8a0-49be-b998-df590094785e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:39.773656 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fae43f-4eac-48ac-b6a0-b39f833065f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac825664b4324c8da293b2aa9c3a9f9e)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:39.775969 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fae43f-4eac-48ac-b6a0-b39f833065f3 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:39.805026 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-ee7a0bf6-d8a0-49be-b998-df590094785e None None] MFA Rules not processed for user `991cdaf69fd5439c9cad12cc2be49ada`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:39.806255 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-ee7a0bf6-d8a0-49be-b998-df590094785e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:39.871263 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1062/2101] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:39.882273 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7a0bf6-d8a0-49be-b998-df590094785e None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:39.883662 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-ee7a0bf6-d8a0-49be-b998-df590094785e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:39.943883 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7a0bf6-d8a0-49be-b998-df590094785e None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:39.947598 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0037dc91-fd50-4900-b7b6-6efb49c53bfd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:39.948210 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037dc91-fd50-4900-b7b6-6efb49c53bfd service cinder] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:39.948498 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037dc91-fd50-4900-b7b6-6efb49c53bfd service cinder] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:39.948780 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037dc91-fd50-4900-b7b6-6efb49c53bfd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:39.957146 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0037dc91-fd50-4900-b7b6-6efb49c53bfd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:39.958576 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0037dc91-fd50-4900-b7b6-6efb49c53bfd service cinder] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:39.967522 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1063/2102] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:21:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:21:40.009197 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-02fae43f-4eac-48ac-b6a0-b39f833065f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac825664b4324c8da293b2aa9c3a9f9e'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:40.009937 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-02fae43f-4eac-48ac-b6a0-b39f833065f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac825664b4324c8da293b2aa9c3a9f9e'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:40.010830 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-02fae43f-4eac-48ac-b6a0-b39f833065f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac825664b4324c8da293b2aa9c3a9f9e'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:40.018313 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-02fae43f-4eac-48ac-b6a0-b39f833065f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac825664b4324c8da293b2aa9c3a9f9e'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:40.033070 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-02fae43f-4eac-48ac-b6a0-b39f833065f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac825664b4324c8da293b2aa9c3a9f9e'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:40.109080 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:40.143423 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1040/2103] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:21:39 2022] DELETE /identity/v3/users/ac825664b4324c8da293b2aa9c3a9f9e => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:40.147939 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c6c0b-1b2e-4d76-aebe-95289553aa04 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:40.148233 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c6c0b-1b2e-4d76-aebe-95289553aa04 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:40.148836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c6c0b-1b2e-4d76-aebe-95289553aa04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:40.167891 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-7d3c6c0b-1b2e-4d76-aebe-95289553aa04 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:40.169107 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-7d3c6c0b-1b2e-4d76-aebe-95289553aa04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:40.234848 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1064/2104] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:40.245246 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fae43f-4eac-48ac-b6a0-b39f833065f3 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:40.247590 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-02fae43f-4eac-48ac-b6a0-b39f833065f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:40.263174 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:40.313183 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fae43f-4eac-48ac-b6a0-b39f833065f3 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:40.317707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc01c601-722e-49f7-967d-b7e941ccca0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:40.318490 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01c601-722e-49f7-967d-b7e941ccca0b service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:40.318771 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01c601-722e-49f7-967d-b7e941ccca0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:40.319045 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01c601-722e-49f7-967d-b7e941ccca0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:40.329608 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc01c601-722e-49f7-967d-b7e941ccca0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:40.331043 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc01c601-722e-49f7-967d-b7e941ccca0b service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:40.340889 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1041/2105] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:21:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:21:40.852771 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c6c0b-1b2e-4d76-aebe-95289553aa04 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:40.854410 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-7d3c6c0b-1b2e-4d76-aebe-95289553aa04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:40.867376 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c6c0b-1b2e-4d76-aebe-95289553aa04 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:40.872582 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf64828-dbbe-4e20-9dfb-87a7600f81eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:40.873299 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf64828-dbbe-4e20-9dfb-87a7600f81eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:40.873682 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf64828-dbbe-4e20-9dfb-87a7600f81eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:40.873959 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf64828-dbbe-4e20-9dfb-87a7600f81eb admin admin] PATH_INFO: `/v3/projects/d9f6439b977648dab4c41480d69a0340` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:40.876745 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf64828-dbbe-4e20-9dfb-87a7600f81eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9f6439b977648dab4c41480d69a0340)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:40.877436 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf64828-dbbe-4e20-9dfb-87a7600f81eb admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:41.058787 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-edf64828-dbbe-4e20-9dfb-87a7600f81eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9f6439b977648dab4c41480d69a0340', 'request_id': 'req-edf64828-dbbe-4e20-9dfb-87a7600f81eb'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:41.064392 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-edf64828-dbbe-4e20-9dfb-87a7600f81eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9f6439b977648dab4c41480d69a0340', 'request_id': 'req-edf64828-dbbe-4e20-9dfb-87a7600f81eb'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:41.165898 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-edf64828-dbbe-4e20-9dfb-87a7600f81eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9f6439b977648dab4c41480d69a0340', 'request_id': 'req-edf64828-dbbe-4e20-9dfb-87a7600f81eb'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:41.179126 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-edf64828-dbbe-4e20-9dfb-87a7600f81eb admin admin] The token cache is being invalidate because project d9f6439b977648dab4c41480d69a0340 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:21:41.179431 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-edf64828-dbbe-4e20-9dfb-87a7600f81eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:41.180100 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-edf64828-dbbe-4e20-9dfb-87a7600f81eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:41.181594 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1065/2106] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:21:40 2022] DELETE /identity/v3/projects/d9f6439b977648dab4c41480d69a0340 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:41.185707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc01c601-722e-49f7-967d-b7e941ccca0b service neutron] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:41.195607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc01c601-722e-49f7-967d-b7e941ccca0b service neutron] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:41.257414 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da6bc8-ded3-42f8-bc04-2badb4509189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:41.258040 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da6bc8-ded3-42f8-bc04-2badb4509189 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:41.258317 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da6bc8-ded3-42f8-bc04-2badb4509189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:41.258618 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da6bc8-ded3-42f8-bc04-2badb4509189 admin admin] PATH_INFO: `/v3/users/0e1e758a21484fc28f31dbc830fc5ee4` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:41.259792 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83da6bc8-ded3-42f8-bc04-2badb4509189 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e1e758a21484fc28f31dbc830fc5ee4)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:41.261810 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83da6bc8-ded3-42f8-bc04-2badb4509189 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:41.473615 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-83da6bc8-ded3-42f8-bc04-2badb4509189 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e1e758a21484fc28f31dbc830fc5ee4'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:41.474347 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-83da6bc8-ded3-42f8-bc04-2badb4509189 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e1e758a21484fc28f31dbc830fc5ee4'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:41.475082 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-83da6bc8-ded3-42f8-bc04-2badb4509189 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e1e758a21484fc28f31dbc830fc5ee4'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:41.482623 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-83da6bc8-ded3-42f8-bc04-2badb4509189 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e1e758a21484fc28f31dbc830fc5ee4'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:41.496704 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-83da6bc8-ded3-42f8-bc04-2badb4509189 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e1e758a21484fc28f31dbc830fc5ee4'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:41.585355 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1042/2107] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:21:41 2022] DELETE /identity/v3/users/0e1e758a21484fc28f31dbc830fc5ee4 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:41.780644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d054e-cdc1-4be7-b01e-93a6ef57bd08 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:41.787840 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d054e-cdc1-4be7-b01e-93a6ef57bd08 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:41.787840 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d054e-cdc1-4be7-b01e-93a6ef57bd08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:41.804592 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-cf7d054e-cdc1-4be7-b01e-93a6ef57bd08 None None] MFA Rules not processed for user `991cdaf69fd5439c9cad12cc2be49ada`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:41.805723 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-cf7d054e-cdc1-4be7-b01e-93a6ef57bd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:41.859539 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:41.872174 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1066/2108] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:42.347984 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da6bc8-ded3-42f8-bc04-2badb4509189 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:42.349533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-83da6bc8-ded3-42f8-bc04-2badb4509189 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:42.400836 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:42.437271 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da6bc8-ded3-42f8-bc04-2badb4509189 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:42.443301 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258327be-b6f7-4dd5-bf9e-c8e5cc0fd87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:42.458762 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258327be-b6f7-4dd5-bf9e-c8e5cc0fd87c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:42.459361 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258327be-b6f7-4dd5-bf9e-c8e5cc0fd87c admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:42.459839 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258327be-b6f7-4dd5-bf9e-c8e5cc0fd87c admin admin] PATH_INFO: `/v3/projects/23de3c57d61244adbb7228a3f9a82223` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:42.461758 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258327be-b6f7-4dd5-bf9e-c8e5cc0fd87c admin admin] RBAC: Authorizing `identity:delete_project(project_id=23de3c57d61244adbb7228a3f9a82223)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:42.464292 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258327be-b6f7-4dd5-bf9e-c8e5cc0fd87c admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:42.603515 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-258327be-b6f7-4dd5-bf9e-c8e5cc0fd87c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23de3c57d61244adbb7228a3f9a82223', 'request_id': 'req-258327be-b6f7-4dd5-bf9e-c8e5cc0fd87c'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:42.611255 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-258327be-b6f7-4dd5-bf9e-c8e5cc0fd87c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23de3c57d61244adbb7228a3f9a82223', 'request_id': 'req-258327be-b6f7-4dd5-bf9e-c8e5cc0fd87c'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:42.727783 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-258327be-b6f7-4dd5-bf9e-c8e5cc0fd87c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23de3c57d61244adbb7228a3f9a82223', 'request_id': 'req-258327be-b6f7-4dd5-bf9e-c8e5cc0fd87c'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:42.741568 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-258327be-b6f7-4dd5-bf9e-c8e5cc0fd87c admin admin] The token cache is being invalidate because project 23de3c57d61244adbb7228a3f9a82223 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:21:42.741869 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-258327be-b6f7-4dd5-bf9e-c8e5cc0fd87c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:42.742796 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-258327be-b6f7-4dd5-bf9e-c8e5cc0fd87c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:42.744219 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1043/2109] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:21:42 2022] DELETE /identity/v3/projects/23de3c57d61244adbb7228a3f9a82223 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:42.749504 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7d054e-cdc1-4be7-b01e-93a6ef57bd08 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:42.761007 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7d054e-cdc1-4be7-b01e-93a6ef57bd08 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:42.818719 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22770572-8153-426e-a8b9-b252ecb52492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:42.819393 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22770572-8153-426e-a8b9-b252ecb52492 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:42.819714 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22770572-8153-426e-a8b9-b252ecb52492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:42.819996 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22770572-8153-426e-a8b9-b252ecb52492 admin admin] PATH_INFO: `/v3/users/0fb6fae0770d42839ee8ff0330faf1ff` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:42.821128 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22770572-8153-426e-a8b9-b252ecb52492 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fb6fae0770d42839ee8ff0330faf1ff)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:42.823052 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22770572-8153-426e-a8b9-b252ecb52492 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:43.008396 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-22770572-8153-426e-a8b9-b252ecb52492 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fb6fae0770d42839ee8ff0330faf1ff'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:43.015552 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-22770572-8153-426e-a8b9-b252ecb52492 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fb6fae0770d42839ee8ff0330faf1ff'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:43.016236 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-22770572-8153-426e-a8b9-b252ecb52492 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fb6fae0770d42839ee8ff0330faf1ff'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:43.028575 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-22770572-8153-426e-a8b9-b252ecb52492 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fb6fae0770d42839ee8ff0330faf1ff'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:43.125868 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-22770572-8153-426e-a8b9-b252ecb52492 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fb6fae0770d42839ee8ff0330faf1ff'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:43.134345 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1067/2110] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:21:42 2022] DELETE /identity/v3/users/0fb6fae0770d42839ee8ff0330faf1ff => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:43.661625 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258327be-b6f7-4dd5-bf9e-c8e5cc0fd87c admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:43.663154 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-258327be-b6f7-4dd5-bf9e-c8e5cc0fd87c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:43.681823 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258327be-b6f7-4dd5-bf9e-c8e5cc0fd87c admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:43.745003 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e44e79-5ce2-4733-9587-99789c75f5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:43.745559 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e44e79-5ce2-4733-9587-99789c75f5b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:43.745847 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e44e79-5ce2-4733-9587-99789c75f5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:43.746145 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e44e79-5ce2-4733-9587-99789c75f5b0 admin admin] PATH_INFO: `/v3/projects/fb0b05729efd4000bd504bc5779d1366` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:43.747300 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e44e79-5ce2-4733-9587-99789c75f5b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb0b05729efd4000bd504bc5779d1366)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:43.751127 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e44e79-5ce2-4733-9587-99789c75f5b0 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:43.916982 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-f5e44e79-5ce2-4733-9587-99789c75f5b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb0b05729efd4000bd504bc5779d1366', 'request_id': 'req-f5e44e79-5ce2-4733-9587-99789c75f5b0'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:43.922605 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-f5e44e79-5ce2-4733-9587-99789c75f5b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb0b05729efd4000bd504bc5779d1366', 'request_id': 'req-f5e44e79-5ce2-4733-9587-99789c75f5b0'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:44.007491 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-f5e44e79-5ce2-4733-9587-99789c75f5b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb0b05729efd4000bd504bc5779d1366', 'request_id': 'req-f5e44e79-5ce2-4733-9587-99789c75f5b0'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:44.008819 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:44.020603 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-f5e44e79-5ce2-4733-9587-99789c75f5b0 admin admin] The token cache is being invalidate because project fb0b05729efd4000bd504bc5779d1366 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:21:44.020914 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-f5e44e79-5ce2-4733-9587-99789c75f5b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:44.021553 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-f5e44e79-5ce2-4733-9587-99789c75f5b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:44.023138 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1044/2111] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:21:43 2022] DELETE /identity/v3/projects/fb0b05729efd4000bd504bc5779d1366 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:44.030105 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cdbc55-b8e2-499f-9c91-3e8c76f6c1ea None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:44.030534 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cdbc55-b8e2-499f-9c91-3e8c76f6c1ea None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:44.040632 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cdbc55-b8e2-499f-9c91-3e8c76f6c1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:44.051629 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-b8cdbc55-b8e2-499f-9c91-3e8c76f6c1ea None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:44.052865 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-b8cdbc55-b8e2-499f-9c91-3e8c76f6c1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:44.121169 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1068/2112] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:44.129605 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef601c97-3c1d-4836-854d-9f6ebcf2362a None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:44.129891 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef601c97-3c1d-4836-854d-9f6ebcf2362a None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:44.130169 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef601c97-3c1d-4836-854d-9f6ebcf2362a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:44.150117 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-ef601c97-3c1d-4836-854d-9f6ebcf2362a None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:44.151349 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-ef601c97-3c1d-4836-854d-9f6ebcf2362a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:44.159921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1045/2113] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:44.164343 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cdbc55-b8e2-499f-9c91-3e8c76f6c1ea None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:44.174912 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cdbc55-b8e2-499f-9c91-3e8c76f6c1ea None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:44.179938 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069c2be-947d-41ef-8871-cba4405c2516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:44.180539 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069c2be-947d-41ef-8871-cba4405c2516 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:44.180825 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069c2be-947d-41ef-8871-cba4405c2516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:44.181108 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069c2be-947d-41ef-8871-cba4405c2516 admin admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:44.181759 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c069c2be-947d-41ef-8871-cba4405c2516 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:21:44.187360 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c069c2be-947d-41ef-8871-cba4405c2516 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:44.187360 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c069c2be-947d-41ef-8871-cba4405c2516 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:44.196204 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1069/2114] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:21:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:21:44.200723 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef601c97-3c1d-4836-854d-9f6ebcf2362a None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:44.214755 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef601c97-3c1d-4836-854d-9f6ebcf2362a None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:44.220673 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe5e87-7d1f-4e05-b50b-1c5988e8bb38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:44.221305 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe5e87-7d1f-4e05-b50b-1c5988e8bb38 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:44.221588 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe5e87-7d1f-4e05-b50b-1c5988e8bb38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:44.221861 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe5e87-7d1f-4e05-b50b-1c5988e8bb38 admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:44.222540 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fe5e87-7d1f-4e05-b50b-1c5988e8bb38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:44.224536 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fe5e87-7d1f-4e05-b50b-1c5988e8bb38 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:44.331601 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1046/2115] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:21:44 2022] POST /identity/v3/projects => generated 379 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:21:44.332841 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069c2be-947d-41ef-8871-cba4405c2516 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:44.338442 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:44.344720 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069c2be-947d-41ef-8871-cba4405c2516 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:44.418845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a623d0-d64d-4975-aa79-95ad4652c97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:44.419518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a623d0-d64d-4975-aa79-95ad4652c97c admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:44.419802 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a623d0-d64d-4975-aa79-95ad4652c97c admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:44.420082 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a623d0-d64d-4975-aa79-95ad4652c97c admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:44.420753 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a623d0-d64d-4975-aa79-95ad4652c97c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:44.422758 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a623d0-d64d-4975-aa79-95ad4652c97c admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:44.545418 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1070/2116] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:21:44 2022] POST /identity/v3/users => generated 290 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:21:44.551321 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe5e87-7d1f-4e05-b50b-1c5988e8bb38 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:44.564606 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe5e87-7d1f-4e05-b50b-1c5988e8bb38 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:44.572776 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be099852-ed00-4dd3-9e49-c964efe8adb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:44.573373 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be099852-ed00-4dd3-9e49-c964efe8adb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:44.573669 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be099852-ed00-4dd3-9e49-c964efe8adb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:44.574007 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be099852-ed00-4dd3-9e49-c964efe8adb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:44.574683 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be099852-ed00-4dd3-9e49-c964efe8adb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:44.577385 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be099852-ed00-4dd3-9e49-c964efe8adb0 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:44.601701 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1047/2117] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:21:44.603392 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a623d0-d64d-4975-aa79-95ad4652c97c admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:44.619258 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a623d0-d64d-4975-aa79-95ad4652c97c admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:44.630116 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd7e25-8185-4d26-8c30-4c3093191f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:44.630722 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd7e25-8185-4d26-8c30-4c3093191f05 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:44.631002 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd7e25-8185-4d26-8c30-4c3093191f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:44.631303 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd7e25-8185-4d26-8c30-4c3093191f05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:44.632016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd7e25-8185-4d26-8c30-4c3093191f05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:44.633607 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd7e25-8185-4d26-8c30-4c3093191f05 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:44.645387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1071/2118] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:21:44.649768 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be099852-ed00-4dd3-9e49-c964efe8adb0 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:44.659987 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be099852-ed00-4dd3-9e49-c964efe8adb0 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:44.664695 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c56ac-b6c1-4ea3-ae4e-64b8622cf4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:44.665345 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c56ac-b6c1-4ea3-ae4e-64b8622cf4da admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:44.665657 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c56ac-b6c1-4ea3-ae4e-64b8622cf4da admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:44.665942 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c56ac-b6c1-4ea3-ae4e-64b8622cf4da admin admin] PATH_INFO: `/v3/projects/b7eda3e14f5c4576b6e510ed06a784f5/users/c1c4c75b8f134464bf00cadfaa67fbe7/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:44.681617 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7c56ac-b6c1-4ea3-ae4e-64b8622cf4da admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7eda3e14f5c4576b6e510ed06a784f5, user_id=c1c4c75b8f134464bf00cadfaa67fbe7, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:44.684375 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7c56ac-b6c1-4ea3-ae4e-64b8622cf4da admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:44.746449 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1048/2119] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:21:44 2022] PUT /identity/v3/projects/b7eda3e14f5c4576b6e510ed06a784f5/users/c1c4c75b8f134464bf00cadfaa67fbe7/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:44.759237 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528f123-2df2-42d6-935b-c5fde1f1d373 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:44.759562 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528f123-2df2-42d6-935b-c5fde1f1d373 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:44.759838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528f123-2df2-42d6-935b-c5fde1f1d373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:44.788562 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-6528f123-2df2-42d6-935b-c5fde1f1d373 None None] MFA Rules not processed for user `c1c4c75b8f134464bf00cadfaa67fbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:44.789798 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-6528f123-2df2-42d6-935b-c5fde1f1d373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:44.861527 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1072/2120] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:44.866207 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c56ac-b6c1-4ea3-ae4e-64b8622cf4da admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:44.954306 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c56ac-b6c1-4ea3-ae4e-64b8622cf4da admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:44.959667 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee62d9f-5ef8-4e26-a7bc-f3e3134d2601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:44.960444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee62d9f-5ef8-4e26-a7bc-f3e3134d2601 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:44.960847 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee62d9f-5ef8-4e26-a7bc-f3e3134d2601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:44.961211 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee62d9f-5ef8-4e26-a7bc-f3e3134d2601 admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:44.961846 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee62d9f-5ef8-4e26-a7bc-f3e3134d2601 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:44.964140 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee62d9f-5ef8-4e26-a7bc-f3e3134d2601 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:45.077693 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1049/2121] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:21:44 2022] POST /identity/v3/projects => generated 377 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:21:45.081956 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528f123-2df2-42d6-935b-c5fde1f1d373 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:45.092857 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528f123-2df2-42d6-935b-c5fde1f1d373 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:45.153461 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef3747-62a8-46bf-a935-7627bb677c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:45.154081 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef3747-62a8-46bf-a935-7627bb677c29 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:45.155136 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef3747-62a8-46bf-a935-7627bb677c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:45.155449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef3747-62a8-46bf-a935-7627bb677c29 admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:45.156247 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ef3747-62a8-46bf-a935-7627bb677c29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:45.158272 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ef3747-62a8-46bf-a935-7627bb677c29 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:45.243690 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:45.250162 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1073/2122] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:21:45 2022] POST /identity/v3/users => generated 289 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:21:45.254092 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee62d9f-5ef8-4e26-a7bc-f3e3134d2601 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:45.265418 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee62d9f-5ef8-4e26-a7bc-f3e3134d2601 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:45.276210 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee9564-d425-4ba5-a3a9-10ffe64c1dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:45.276770 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee9564-d425-4ba5-a3a9-10ffe64c1dc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:45.286250 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee9564-d425-4ba5-a3a9-10ffe64c1dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:45.286250 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee9564-d425-4ba5-a3a9-10ffe64c1dc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:45.286250 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acee9564-d425-4ba5-a3a9-10ffe64c1dc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:45.286250 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acee9564-d425-4ba5-a3a9-10ffe64c1dc2 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:45.291853 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1050/2123] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:21:45.296193 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef3747-62a8-46bf-a935-7627bb677c29 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:45.306895 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef3747-62a8-46bf-a935-7627bb677c29 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:45.312169 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369c7d47-883f-4ccb-9fe1-421e351b8d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:45.312770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c7d47-883f-4ccb-9fe1-421e351b8d99 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:45.313053 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c7d47-883f-4ccb-9fe1-421e351b8d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:45.314903 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c7d47-883f-4ccb-9fe1-421e351b8d99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:45.314903 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369c7d47-883f-4ccb-9fe1-421e351b8d99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:45.315711 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369c7d47-883f-4ccb-9fe1-421e351b8d99 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:45.326877 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1074/2124] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 12:21:45.330906 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee9564-d425-4ba5-a3a9-10ffe64c1dc2 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:45.340985 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee9564-d425-4ba5-a3a9-10ffe64c1dc2 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:45.345581 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4457a0ce-ec33-488a-8791-7bf6804210d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:45.346232 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457a0ce-ec33-488a-8791-7bf6804210d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:45.346555 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457a0ce-ec33-488a-8791-7bf6804210d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:45.346832 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457a0ce-ec33-488a-8791-7bf6804210d7 admin admin] PATH_INFO: `/v3/projects/a4e9af24237243a0ac501918e0abec63/users/ee410fa04abc48fca7b05938b5e14f7b/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:45.362473 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4457a0ce-ec33-488a-8791-7bf6804210d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4e9af24237243a0ac501918e0abec63, user_id=ee410fa04abc48fca7b05938b5e14f7b, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:45.365203 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4457a0ce-ec33-488a-8791-7bf6804210d7 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:45.469654 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1051/2125] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:21:45 2022] PUT /identity/v3/projects/a4e9af24237243a0ac501918e0abec63/users/ee410fa04abc48fca7b05938b5e14f7b/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:45.482127 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23e2c1-0e3a-4f4d-904a-40669d876545 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:45.483389 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23e2c1-0e3a-4f4d-904a-40669d876545 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:45.483502 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23e2c1-0e3a-4f4d-904a-40669d876545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:45.514583 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-ff23e2c1-0e3a-4f4d-904a-40669d876545 None None] MFA Rules not processed for user `ee410fa04abc48fca7b05938b5e14f7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:45.515838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-ff23e2c1-0e3a-4f4d-904a-40669d876545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:45.593028 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1075/2126] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:21:45.597725 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4457a0ce-ec33-488a-8791-7bf6804210d7 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:45.674174 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:45.698361 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4457a0ce-ec33-488a-8791-7bf6804210d7 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:45.705370 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61de349-2010-45f5-a549-47927fc1a3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:45.705994 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61de349-2010-45f5-a549-47927fc1a3dc admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:45.706299 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61de349-2010-45f5-a549-47927fc1a3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:45.706556 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61de349-2010-45f5-a549-47927fc1a3dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:45.707208 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61de349-2010-45f5-a549-47927fc1a3dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:45.709146 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61de349-2010-45f5-a549-47927fc1a3dc admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:45.985149 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1052/2127] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:21:45 2022] POST /identity/v3/projects => generated 377 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:21:45.989256 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23e2c1-0e3a-4f4d-904a-40669d876545 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:46.016173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23e2c1-0e3a-4f4d-904a-40669d876545 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:46.086503 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3596a6c6-8d44-4344-ba95-b77b3f2c9a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:46.087900 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3596a6c6-8d44-4344-ba95-b77b3f2c9a1f admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:46.088340 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3596a6c6-8d44-4344-ba95-b77b3f2c9a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:46.088772 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3596a6c6-8d44-4344-ba95-b77b3f2c9a1f admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:46.089572 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3596a6c6-8d44-4344-ba95-b77b3f2c9a1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:46.092391 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3596a6c6-8d44-4344-ba95-b77b3f2c9a1f admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:46.253899 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1076/2128] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:21:45 2022] POST /identity/v3/users => generated 295 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:21:46.257832 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61de349-2010-45f5-a549-47927fc1a3dc admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:46.268875 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61de349-2010-45f5-a549-47927fc1a3dc admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:46.274124 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c24871-138a-497b-8fbe-c1e781b7d5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:46.274710 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c24871-138a-497b-8fbe-c1e781b7d5d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:46.274988 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c24871-138a-497b-8fbe-c1e781b7d5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:46.275265 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c24871-138a-497b-8fbe-c1e781b7d5d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:46.275998 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c24871-138a-497b-8fbe-c1e781b7d5d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:46.277670 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c24871-138a-497b-8fbe-c1e781b7d5d4 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:46.290570 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1053/2129] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:21:46.295882 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3596a6c6-8d44-4344-ba95-b77b3f2c9a1f admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:46.313221 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3596a6c6-8d44-4344-ba95-b77b3f2c9a1f admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:46.319008 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e81229-955c-4e29-a338-982874eed96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:46.320019 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e81229-955c-4e29-a338-982874eed96c admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:46.320344 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e81229-955c-4e29-a338-982874eed96c admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:46.320655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e81229-955c-4e29-a338-982874eed96c admin admin] PATH_INFO: `/v3/projects/c1b733bd85504812bdfb51ac3c8f27eb/users/680577a08d2048a9985c196f6529ea25/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:46.345672 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e81229-955c-4e29-a338-982874eed96c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1b733bd85504812bdfb51ac3c8f27eb, user_id=680577a08d2048a9985c196f6529ea25, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:46.345672 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e81229-955c-4e29-a338-982874eed96c admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:46.521392 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1077/2130] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:21:46 2022] PUT /identity/v3/projects/c1b733bd85504812bdfb51ac3c8f27eb/users/680577a08d2048a9985c196f6529ea25/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:46.522462 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c24871-138a-497b-8fbe-c1e781b7d5d4 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:46.533453 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c24871-138a-497b-8fbe-c1e781b7d5d4 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:46.594803 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8716318-39ba-4052-bc11-e59d72031431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:46.595483 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8716318-39ba-4052-bc11-e59d72031431 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:46.595769 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8716318-39ba-4052-bc11-e59d72031431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:46.596045 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8716318-39ba-4052-bc11-e59d72031431 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:46.596735 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8716318-39ba-4052-bc11-e59d72031431 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:46.598410 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8716318-39ba-4052-bc11-e59d72031431 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:46.609278 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1054/2131] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:21:46.613570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e81229-955c-4e29-a338-982874eed96c admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:46.630905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e81229-955c-4e29-a338-982874eed96c admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:46.639675 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e1e455-aa54-48a7-b05f-fcc24c8652a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:46.640367 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1e455-aa54-48a7-b05f-fcc24c8652a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:46.640683 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1e455-aa54-48a7-b05f-fcc24c8652a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:46.640987 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1e455-aa54-48a7-b05f-fcc24c8652a1 admin admin] PATH_INFO: `/v3/projects/c1b733bd85504812bdfb51ac3c8f27eb/users/680577a08d2048a9985c196f6529ea25/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:46.643009 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e1e455-aa54-48a7-b05f-fcc24c8652a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1b733bd85504812bdfb51ac3c8f27eb, user_id=680577a08d2048a9985c196f6529ea25, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:46.645741 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e1e455-aa54-48a7-b05f-fcc24c8652a1 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:46.777245 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1078/2132] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:21:46 2022] PUT /identity/v3/projects/c1b733bd85504812bdfb51ac3c8f27eb/users/680577a08d2048a9985c196f6529ea25/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:46.787740 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e55fcd-0a64-492e-ad34-009de7cdbc0f None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:46.788311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e55fcd-0a64-492e-ad34-009de7cdbc0f None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:46.788311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e55fcd-0a64-492e-ad34-009de7cdbc0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:46.845536 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-45e55fcd-0a64-492e-ad34-009de7cdbc0f None None] MFA Rules not processed for user `680577a08d2048a9985c196f6529ea25`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:46.847348 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-45e55fcd-0a64-492e-ad34-009de7cdbc0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:46.933356 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1055/2133] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:46 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:46.945746 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e1e455-aa54-48a7-b05f-fcc24c8652a1 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:46.947499 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-52e1e455-aa54-48a7-b05f-fcc24c8652a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:46.998317 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:47.022967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e1e455-aa54-48a7-b05f-fcc24c8652a1 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:47.027879 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa560429-0eb2-423a-bf27-3994a19f5222 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:47.028633 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa560429-0eb2-423a-bf27-3994a19f5222 service cinder] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:47.029003 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa560429-0eb2-423a-bf27-3994a19f5222 service cinder] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:47.029907 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa560429-0eb2-423a-bf27-3994a19f5222 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:47.040623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa560429-0eb2-423a-bf27-3994a19f5222 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:47.042055 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa560429-0eb2-423a-bf27-3994a19f5222 service cinder] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:47.055247 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1079/2134] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:21:46 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:21:47.176995 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e55fcd-0a64-492e-ad34-009de7cdbc0f None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:47.180358 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-45e55fcd-0a64-492e-ad34-009de7cdbc0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:47.195133 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e55fcd-0a64-492e-ad34-009de7cdbc0f None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:47.198956 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f63d49-efa1-4e7b-b2c4-afaf74d037f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:47.199711 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f63d49-efa1-4e7b-b2c4-afaf74d037f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:47.199995 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f63d49-efa1-4e7b-b2c4-afaf74d037f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:47.200480 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f63d49-efa1-4e7b-b2c4-afaf74d037f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:47.241110 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:47.269846 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f63d49-efa1-4e7b-b2c4-afaf74d037f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:47.271372 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f63d49-efa1-4e7b-b2c4-afaf74d037f9 service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:47.281942 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1056/2135] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:21:47 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:21:55.068204 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa560429-0eb2-423a-bf27-3994a19f5222 service cinder] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:55.139030 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa560429-0eb2-423a-bf27-3994a19f5222 service cinder] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:55.146892 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aba11d-3afd-4844-8baa-761d85a095e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:55.147585 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aba11d-3afd-4844-8baa-761d85a095e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:55.147878 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aba11d-3afd-4844-8baa-761d85a095e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:55.148185 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aba11d-3afd-4844-8baa-761d85a095e9 admin admin] PATH_INFO: `/v3/users/c1c4c75b8f134464bf00cadfaa67fbe7` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:55.149341 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aba11d-3afd-4844-8baa-761d85a095e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1c4c75b8f134464bf00cadfaa67fbe7)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:55.151264 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aba11d-3afd-4844-8baa-761d85a095e9 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:55.431430 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-62aba11d-3afd-4844-8baa-761d85a095e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1c4c75b8f134464bf00cadfaa67fbe7'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:55.449634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-62aba11d-3afd-4844-8baa-761d85a095e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1c4c75b8f134464bf00cadfaa67fbe7'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:55.452982 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-62aba11d-3afd-4844-8baa-761d85a095e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1c4c75b8f134464bf00cadfaa67fbe7'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:55.473882 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-62aba11d-3afd-4844-8baa-761d85a095e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1c4c75b8f134464bf00cadfaa67fbe7'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:55.622023 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-62aba11d-3afd-4844-8baa-761d85a095e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1c4c75b8f134464bf00cadfaa67fbe7'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:55.629380 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1080/2136] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:21:55 2022] DELETE /identity/v3/users/c1c4c75b8f134464bf00cadfaa67fbe7 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:55.634801 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715f286-01e6-4556-b55f-e055fd301ba9 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:55.635442 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715f286-01e6-4556-b55f-e055fd301ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:55.635442 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715f286-01e6-4556-b55f-e055fd301ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:55.654224 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-8715f286-01e6-4556-b55f-e055fd301ba9 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:55.655461 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-8715f286-01e6-4556-b55f-e055fd301ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:55.733757 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1057/2137] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:55.744670 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aba11d-3afd-4844-8baa-761d85a095e9 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:55.746912 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-62aba11d-3afd-4844-8baa-761d85a095e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:55.823821 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aba11d-3afd-4844-8baa-761d85a095e9 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:55.827646 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276aec4d-6bfd-443d-aa5d-91bcbebed8d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:55.828301 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276aec4d-6bfd-443d-aa5d-91bcbebed8d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:55.828586 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276aec4d-6bfd-443d-aa5d-91bcbebed8d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:55.828866 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276aec4d-6bfd-443d-aa5d-91bcbebed8d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:55.836784 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:55.839474 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276aec4d-6bfd-443d-aa5d-91bcbebed8d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:55.841005 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276aec4d-6bfd-443d-aa5d-91bcbebed8d4 service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:55.853349 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1081/2138] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:21:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:21:56.455131 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8715f286-01e6-4556-b55f-e055fd301ba9 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:56.456722 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-8715f286-01e6-4556-b55f-e055fd301ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:56.476565 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8715f286-01e6-4556-b55f-e055fd301ba9 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:56.489128 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81043e15-6fdd-467e-bca6-c79382b3944b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:56.490959 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81043e15-6fdd-467e-bca6-c79382b3944b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:56.491529 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81043e15-6fdd-467e-bca6-c79382b3944b admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:56.491964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81043e15-6fdd-467e-bca6-c79382b3944b admin admin] PATH_INFO: `/v3/projects/b7eda3e14f5c4576b6e510ed06a784f5` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:56.494838 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81043e15-6fdd-467e-bca6-c79382b3944b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7eda3e14f5c4576b6e510ed06a784f5)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:56.498243 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81043e15-6fdd-467e-bca6-c79382b3944b admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:56.749549 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-81043e15-6fdd-467e-bca6-c79382b3944b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7eda3e14f5c4576b6e510ed06a784f5', 'request_id': 'req-81043e15-6fdd-467e-bca6-c79382b3944b'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:56.765097 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-81043e15-6fdd-467e-bca6-c79382b3944b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7eda3e14f5c4576b6e510ed06a784f5', 'request_id': 'req-81043e15-6fdd-467e-bca6-c79382b3944b'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:56.850653 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-81043e15-6fdd-467e-bca6-c79382b3944b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7eda3e14f5c4576b6e510ed06a784f5', 'request_id': 'req-81043e15-6fdd-467e-bca6-c79382b3944b'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:56.869011 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-81043e15-6fdd-467e-bca6-c79382b3944b admin admin] The token cache is being invalidate because project b7eda3e14f5c4576b6e510ed06a784f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:21:56.869579 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-81043e15-6fdd-467e-bca6-c79382b3944b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:56.873901 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-81043e15-6fdd-467e-bca6-c79382b3944b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:56.876207 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1058/2139] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:21:56 2022] DELETE /identity/v3/projects/b7eda3e14f5c4576b6e510ed06a784f5 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:21:56.881785 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276aec4d-6bfd-443d-aa5d-91bcbebed8d4 service neutron] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:56.897463 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276aec4d-6bfd-443d-aa5d-91bcbebed8d4 service neutron] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:57.008439 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c82b8-6aa8-4a96-99c9-6508cbef6fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:57.009079 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c82b8-6aa8-4a96-99c9-6508cbef6fb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:57.012910 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c82b8-6aa8-4a96-99c9-6508cbef6fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:57.012910 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c82b8-6aa8-4a96-99c9-6508cbef6fb7 admin admin] PATH_INFO: `/v3/users/ee410fa04abc48fca7b05938b5e14f7b` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:57.012910 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c82b8-6aa8-4a96-99c9-6508cbef6fb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee410fa04abc48fca7b05938b5e14f7b)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:57.013186 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c82b8-6aa8-4a96-99c9-6508cbef6fb7 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:57.220361 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-030c82b8-6aa8-4a96-99c9-6508cbef6fb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee410fa04abc48fca7b05938b5e14f7b'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:57.231736 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-030c82b8-6aa8-4a96-99c9-6508cbef6fb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee410fa04abc48fca7b05938b5e14f7b'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:57.233141 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-030c82b8-6aa8-4a96-99c9-6508cbef6fb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee410fa04abc48fca7b05938b5e14f7b'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:57.260955 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-030c82b8-6aa8-4a96-99c9-6508cbef6fb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee410fa04abc48fca7b05938b5e14f7b'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:57.303050 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81043e15-6fdd-467e-bca6-c79382b3944b admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:57.304664 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-81043e15-6fdd-467e-bca6-c79382b3944b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:57.323858 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81043e15-6fdd-467e-bca6-c79382b3944b admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:57.331140 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09fbdad-4925-4341-9e4e-adc9799252dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:57.332207 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09fbdad-4925-4341-9e4e-adc9799252dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:57.332493 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09fbdad-4925-4341-9e4e-adc9799252dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:57.332781 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09fbdad-4925-4341-9e4e-adc9799252dd admin admin] PATH_INFO: `/v3/users/2ac6813dae8b4496a8a5f5ef98407530` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:57.343237 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09fbdad-4925-4341-9e4e-adc9799252dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ac6813dae8b4496a8a5f5ef98407530)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:57.345123 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09fbdad-4925-4341-9e4e-adc9799252dd admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:57.363929 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-030c82b8-6aa8-4a96-99c9-6508cbef6fb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee410fa04abc48fca7b05938b5e14f7b'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:57.371598 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1082/2140] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:21:56 2022] DELETE /identity/v3/users/ee410fa04abc48fca7b05938b5e14f7b => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:57.640460 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-b09fbdad-4925-4341-9e4e-adc9799252dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ac6813dae8b4496a8a5f5ef98407530'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:57.642237 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-b09fbdad-4925-4341-9e4e-adc9799252dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ac6813dae8b4496a8a5f5ef98407530'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:57.643429 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-b09fbdad-4925-4341-9e4e-adc9799252dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ac6813dae8b4496a8a5f5ef98407530'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:57.649165 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:57.654159 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-b09fbdad-4925-4341-9e4e-adc9799252dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ac6813dae8b4496a8a5f5ef98407530'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:57.668206 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-b09fbdad-4925-4341-9e4e-adc9799252dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ac6813dae8b4496a8a5f5ef98407530'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:57.779016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1059/2141] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:21:57 2022] DELETE /identity/v3/users/2ac6813dae8b4496a8a5f5ef98407530 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:21:57.788197 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc0d60-f0c9-4faa-93e3-48e64f4d6b85 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:57.788670 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc0d60-f0c9-4faa-93e3-48e64f4d6b85 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:57.789231 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc0d60-f0c9-4faa-93e3-48e64f4d6b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:57.815163 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-48cc0d60-f0c9-4faa-93e3-48e64f4d6b85 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:57.816463 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-48cc0d60-f0c9-4faa-93e3-48e64f4d6b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:57.903852 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09fbdad-4925-4341-9e4e-adc9799252dd admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:57.906313 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-b09fbdad-4925-4341-9e4e-adc9799252dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:57.913366 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1083/2142] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:57.926478 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc0d60-f0c9-4faa-93e3-48e64f4d6b85 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:57.929010 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-48cc0d60-f0c9-4faa-93e3-48e64f4d6b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:57.932865 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09fbdad-4925-4341-9e4e-adc9799252dd admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:57.975360 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62319d85-c50f-40c2-8f3a-1ce4f2e9eae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:57.976362 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:57.978445 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62319d85-c50f-40c2-8f3a-1ce4f2e9eae5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:57.982741 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62319d85-c50f-40c2-8f3a-1ce4f2e9eae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:57.986494 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62319d85-c50f-40c2-8f3a-1ce4f2e9eae5 admin admin] PATH_INFO: `/v3/projects/a4e9af24237243a0ac501918e0abec63` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:57.993816 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62319d85-c50f-40c2-8f3a-1ce4f2e9eae5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4e9af24237243a0ac501918e0abec63)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:57.998585 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62319d85-c50f-40c2-8f3a-1ce4f2e9eae5 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:58.068939 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc0d60-f0c9-4faa-93e3-48e64f4d6b85 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:58.077096 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87397c8b-7af3-4833-b90b-75189ffbc03c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:58.077879 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87397c8b-7af3-4833-b90b-75189ffbc03c service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:58.078171 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87397c8b-7af3-4833-b90b-75189ffbc03c service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:58.078450 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87397c8b-7af3-4833-b90b-75189ffbc03c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:58.089335 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87397c8b-7af3-4833-b90b-75189ffbc03c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:58.090791 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87397c8b-7af3-4833-b90b-75189ffbc03c service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:58.109928 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1084/2143] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:21:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:21:58.119632 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87397c8b-7af3-4833-b90b-75189ffbc03c service neutron] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:58.121849 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-87397c8b-7af3-4833-b90b-75189ffbc03c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:58.145685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87397c8b-7af3-4833-b90b-75189ffbc03c service neutron] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:58.155656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58a64fe-b5c8-46af-9444-18e6b906b4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:58.157978 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58a64fe-b5c8-46af-9444-18e6b906b4a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:58.158387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58a64fe-b5c8-46af-9444-18e6b906b4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:58.158801 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58a64fe-b5c8-46af-9444-18e6b906b4a8 admin admin] PATH_INFO: `/v3/users/67d48cb065e9467e91a914742513fb01` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:58.160226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58a64fe-b5c8-46af-9444-18e6b906b4a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67d48cb065e9467e91a914742513fb01)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:58.163166 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58a64fe-b5c8-46af-9444-18e6b906b4a8 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:58.214876 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-62319d85-c50f-40c2-8f3a-1ce4f2e9eae5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4e9af24237243a0ac501918e0abec63', 'request_id': 'req-62319d85-c50f-40c2-8f3a-1ce4f2e9eae5'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:58.228786 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-62319d85-c50f-40c2-8f3a-1ce4f2e9eae5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4e9af24237243a0ac501918e0abec63', 'request_id': 'req-62319d85-c50f-40c2-8f3a-1ce4f2e9eae5'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:58.352500 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-62319d85-c50f-40c2-8f3a-1ce4f2e9eae5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4e9af24237243a0ac501918e0abec63', 'request_id': 'req-62319d85-c50f-40c2-8f3a-1ce4f2e9eae5'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:58.366732 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-62319d85-c50f-40c2-8f3a-1ce4f2e9eae5 admin admin] The token cache is being invalidate because project a4e9af24237243a0ac501918e0abec63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:21:58.367027 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-62319d85-c50f-40c2-8f3a-1ce4f2e9eae5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:58.367727 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-62319d85-c50f-40c2-8f3a-1ce4f2e9eae5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:58.369499 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1060/2144] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:21:57 2022] DELETE /identity/v3/projects/a4e9af24237243a0ac501918e0abec63 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:58.373793 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62319d85-c50f-40c2-8f3a-1ce4f2e9eae5 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:58.375144 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-62319d85-c50f-40c2-8f3a-1ce4f2e9eae5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:58.399143 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-c58a64fe-b5c8-46af-9444-18e6b906b4a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67d48cb065e9467e91a914742513fb01'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:58.409896 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-c58a64fe-b5c8-46af-9444-18e6b906b4a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67d48cb065e9467e91a914742513fb01'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:58.411546 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-c58a64fe-b5c8-46af-9444-18e6b906b4a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67d48cb065e9467e91a914742513fb01'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:58.438070 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-c58a64fe-b5c8-46af-9444-18e6b906b4a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67d48cb065e9467e91a914742513fb01'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:58.471213 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62319d85-c50f-40c2-8f3a-1ce4f2e9eae5 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:58.488431 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ddadc-9d98-45a4-9412-ede867c7ab28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:58.490063 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ddadc-9d98-45a4-9412-ede867c7ab28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:58.490444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ddadc-9d98-45a4-9412-ede867c7ab28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:58.490823 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ddadc-9d98-45a4-9412-ede867c7ab28 admin admin] PATH_INFO: `/v3/users/680577a08d2048a9985c196f6529ea25` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:58.492291 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ddadc-9d98-45a4-9412-ede867c7ab28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=680577a08d2048a9985c196f6529ea25)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:58.494663 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ddadc-9d98-45a4-9412-ede867c7ab28 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:58.520069 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-c58a64fe-b5c8-46af-9444-18e6b906b4a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67d48cb065e9467e91a914742513fb01'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:58.532466 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1085/2145] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:21:58 2022] DELETE /identity/v3/users/67d48cb065e9467e91a914742513fb01 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:21:58.535344 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a4791-178d-435e-9fac-4ef5830df329 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:58.535756 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a4791-178d-435e-9fac-4ef5830df329 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:58.536037 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a4791-178d-435e-9fac-4ef5830df329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:58.566314 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-427a4791-178d-435e-9fac-4ef5830df329 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:58.567996 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-427a4791-178d-435e-9fac-4ef5830df329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:58.677294 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1086/2146] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:58.695603 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a4791-178d-435e-9fac-4ef5830df329 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:58.699402 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-427a4791-178d-435e-9fac-4ef5830df329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:58.753303 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-072ddadc-9d98-45a4-9412-ede867c7ab28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '680577a08d2048a9985c196f6529ea25'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:58.755007 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-072ddadc-9d98-45a4-9412-ede867c7ab28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '680577a08d2048a9985c196f6529ea25'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:58.755831 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-072ddadc-9d98-45a4-9412-ede867c7ab28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '680577a08d2048a9985c196f6529ea25'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:58.770947 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-072ddadc-9d98-45a4-9412-ede867c7ab28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '680577a08d2048a9985c196f6529ea25'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:58.788192 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-072ddadc-9d98-45a4-9412-ede867c7ab28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '680577a08d2048a9985c196f6529ea25'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:58.855304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a4791-178d-435e-9fac-4ef5830df329 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:58.862419 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724b560-25ac-429a-a1db-49ff1a76e039 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:58.863114 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724b560-25ac-429a-a1db-49ff1a76e039 service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:58.863544 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724b560-25ac-429a-a1db-49ff1a76e039 service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:58.864710 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724b560-25ac-429a-a1db-49ff1a76e039 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:58.866995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-2724b560-25ac-429a-a1db-49ff1a76e039 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:58.882462 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1061/2147] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:21:58 2022] DELETE /identity/v3/users/680577a08d2048a9985c196f6529ea25 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:58.885689 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2724b560-25ac-429a-a1db-49ff1a76e039 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:58.886207 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2724b560-25ac-429a-a1db-49ff1a76e039 service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:58.887097 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ddadc-9d98-45a4-9412-ede867c7ab28 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:58.891073 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-072ddadc-9d98-45a4-9412-ede867c7ab28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:58.898788 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1087/2148] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:21:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:21:58.981090 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:58.993571 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ddadc-9d98-45a4-9412-ede867c7ab28 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:59.002567 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a13e59-5492-4f86-b053-079e480ab687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:59.003624 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a13e59-5492-4f86-b053-079e480ab687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:59.005972 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a13e59-5492-4f86-b053-079e480ab687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:59.005972 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a13e59-5492-4f86-b053-079e480ab687 admin admin] PATH_INFO: `/v3/projects/edf75c0cdb814a1cb74e33660e16429c` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:59.005972 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a13e59-5492-4f86-b053-079e480ab687 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edf75c0cdb814a1cb74e33660e16429c)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:59.008013 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a13e59-5492-4f86-b053-079e480ab687 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:59.230822 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-75a13e59-5492-4f86-b053-079e480ab687 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edf75c0cdb814a1cb74e33660e16429c', 'request_id': 'req-75a13e59-5492-4f86-b053-079e480ab687'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:59.256582 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-75a13e59-5492-4f86-b053-079e480ab687 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edf75c0cdb814a1cb74e33660e16429c', 'request_id': 'req-75a13e59-5492-4f86-b053-079e480ab687'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:59.367445 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-75a13e59-5492-4f86-b053-079e480ab687 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edf75c0cdb814a1cb74e33660e16429c', 'request_id': 'req-75a13e59-5492-4f86-b053-079e480ab687'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:59.415005 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-75a13e59-5492-4f86-b053-079e480ab687 admin admin] The token cache is being invalidate because project edf75c0cdb814a1cb74e33660e16429c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:21:59.415005 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-75a13e59-5492-4f86-b053-079e480ab687 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:59.418307 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-75a13e59-5492-4f86-b053-079e480ab687 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:21:59.421170 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1062/2149] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:21:58 2022] DELETE /identity/v3/projects/edf75c0cdb814a1cb74e33660e16429c => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:21:59.427284 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30c089-0dd1-493b-a1a0-ff8332bbbf78 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:59.427853 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30c089-0dd1-493b-a1a0-ff8332bbbf78 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:59.427998 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30c089-0dd1-493b-a1a0-ff8332bbbf78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:59.455605 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-dc30c089-0dd1-493b-a1a0-ff8332bbbf78 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:59.457476 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-dc30c089-0dd1-493b-a1a0-ff8332bbbf78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:59.462580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:21:59.608248 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1088/2150] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:59.616189 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f48f0-ed4c-41d9-8f95-225ed724b86a None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:59.617471 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f48f0-ed4c-41d9-8f95-225ed724b86a None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:59.617471 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f48f0-ed4c-41d9-8f95-225ed724b86a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:59.640391 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-8f6f48f0-ed4c-41d9-8f95-225ed724b86a None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:21:59.642181 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-8f6f48f0-ed4c-41d9-8f95-225ed724b86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:21:59.663211 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1063/2151] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:21:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:21:59.668066 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30c089-0dd1-493b-a1a0-ff8332bbbf78 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:59.681802 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30c089-0dd1-493b-a1a0-ff8332bbbf78 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:59.686845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b047b-2558-435c-a7e0-6dd9cb416622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:59.688758 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b047b-2558-435c-a7e0-6dd9cb416622 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:59.689082 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b047b-2558-435c-a7e0-6dd9cb416622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:59.689403 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b047b-2558-435c-a7e0-6dd9cb416622 admin admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:59.690585 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b047b-2558-435c-a7e0-6dd9cb416622 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:21:59.691115 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b047b-2558-435c-a7e0-6dd9cb416622 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:59.694267 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b047b-2558-435c-a7e0-6dd9cb416622 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:59.708135 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1089/2152] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:21:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:21:59.715381 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f48f0-ed4c-41d9-8f95-225ed724b86a None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:59.727514 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f48f0-ed4c-41d9-8f95-225ed724b86a None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:59.732573 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f3d4d8-adbb-4e11-b815-d959b490ad19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:59.733195 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3d4d8-adbb-4e11-b815-d959b490ad19 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:59.733585 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3d4d8-adbb-4e11-b815-d959b490ad19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:59.737954 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3d4d8-adbb-4e11-b815-d959b490ad19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:59.737954 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f3d4d8-adbb-4e11-b815-d959b490ad19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:59.737954 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f3d4d8-adbb-4e11-b815-d959b490ad19 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:21:59.870800 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1064/2153] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:21:59 2022] POST /identity/v3/projects => generated 385 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:21:59.873269 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b047b-2558-435c-a7e0-6dd9cb416622 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:21:59.889020 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b047b-2558-435c-a7e0-6dd9cb416622 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:21:59.954208 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9472b30f-a72e-4f8c-895c-7ae5a8887dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:21:59.954933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472b30f-a72e-4f8c-895c-7ae5a8887dab admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:21:59.955286 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472b30f-a72e-4f8c-895c-7ae5a8887dab admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:21:59.955692 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472b30f-a72e-4f8c-895c-7ae5a8887dab admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:21:59.956428 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9472b30f-a72e-4f8c-895c-7ae5a8887dab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:21:59.958862 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9472b30f-a72e-4f8c-895c-7ae5a8887dab admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:00.064323 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a14c05-9cb4-4994-83c7-7e6446ccf75f None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:00.064636 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a14c05-9cb4-4994-83c7-7e6446ccf75f None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:00.064919 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a14c05-9cb4-4994-83c7-7e6446ccf75f None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:00.066524 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1065/2154] 10.222.0.98 () {54 vars in 921 bytes} [Thu Aug 18 12:22:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:22:00.079069 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345eb4e6-7c1f-4a7e-8815-e4ad966f437f None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:00.079069 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345eb4e6-7c1f-4a7e-8815-e4ad966f437f None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:00.079359 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345eb4e6-7c1f-4a7e-8815-e4ad966f437f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:00.109136 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1090/2155] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:21:59 2022] POST /identity/v3/users => generated 293 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:22:00.113214 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9472b30f-a72e-4f8c-895c-7ae5a8887dab admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:00.120267 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-345eb4e6-7c1f-4a7e-8815-e4ad966f437f None None] MFA Rules not processed for user `193637acadc14ec4a48ab79566ba3dab`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:00.121628 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-345eb4e6-7c1f-4a7e-8815-e4ad966f437f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:00.124641 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9472b30f-a72e-4f8c-895c-7ae5a8887dab admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:00.130435 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbef84f-fc10-4d91-81b1-0a6775d446ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:00.131561 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbef84f-fc10-4d91-81b1-0a6775d446ff admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:00.132016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbef84f-fc10-4d91-81b1-0a6775d446ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:00.132657 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbef84f-fc10-4d91-81b1-0a6775d446ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:00.137682 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbef84f-fc10-4d91-81b1-0a6775d446ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:00.137682 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbef84f-fc10-4d91-81b1-0a6775d446ff admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:00.148362 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1091/2156] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:22:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 12:22:00.152514 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbef84f-fc10-4d91-81b1-0a6775d446ff admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:00.172019 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbef84f-fc10-4d91-81b1-0a6775d446ff admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:00.173055 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f49d51-2844-4fe2-8cf7-553a3b1c8fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:00.173655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f49d51-2844-4fe2-8cf7-553a3b1c8fb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:00.174129 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f49d51-2844-4fe2-8cf7-553a3b1c8fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:00.174566 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f49d51-2844-4fe2-8cf7-553a3b1c8fb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:00.175247 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f49d51-2844-4fe2-8cf7-553a3b1c8fb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:00.177038 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f49d51-2844-4fe2-8cf7-553a3b1c8fb2 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:00.187753 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1092/2157] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:22:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:22:00.192132 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f49d51-2844-4fe2-8cf7-553a3b1c8fb2 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:00.199681 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1066/2158] 10.222.0.98 () {58 vars in 1018 bytes} [Thu Aug 18 12:22:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:22:00.202083 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f49d51-2844-4fe2-8cf7-553a3b1c8fb2 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:00.211322 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345eb4e6-7c1f-4a7e-8815-e4ad966f437f None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:00.212157 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013e8fa-a79a-4c5b-96e2-f4072c6f82bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:00.212720 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-345eb4e6-7c1f-4a7e-8815-e4ad966f437f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:00.212822 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013e8fa-a79a-4c5b-96e2-f4072c6f82bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:00.213176 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013e8fa-a79a-4c5b-96e2-f4072c6f82bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:00.213453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013e8fa-a79a-4c5b-96e2-f4072c6f82bd admin admin] PATH_INFO: `/v3/projects/b5e107c5e832436e9a51cec433681a1d/users/0ce4f277d3234c1483a4f850548b514f/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:00.225955 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345eb4e6-7c1f-4a7e-8815-e4ad966f437f None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:00.229257 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3013e8fa-a79a-4c5b-96e2-f4072c6f82bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5e107c5e832436e9a51cec433681a1d, user_id=0ce4f277d3234c1483a4f850548b514f, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:00.231259 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e032ee68-ecf0-46e0-bd29-87e7e3d4b95b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193637acadc14ec4a48ab79566ba3dab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:00.231898 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032ee68-ecf0-46e0-bd29-87e7e3d4b95b service placement] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:00.232188 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032ee68-ecf0-46e0-bd29-87e7e3d4b95b service placement] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:00.232343 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3013e8fa-a79a-4c5b-96e2-f4072c6f82bd admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:00.232488 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032ee68-ecf0-46e0-bd29-87e7e3d4b95b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:00.241327 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e032ee68-ecf0-46e0-bd29-87e7e3d4b95b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:00.242908 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e032ee68-ecf0-46e0-bd29-87e7e3d4b95b service placement] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:00.253565 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1067/2159] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:22:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:22:00.282381 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e032ee68-ecf0-46e0-bd29-87e7e3d4b95b service placement] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:00.283927 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-e032ee68-ecf0-46e0-bd29-87e7e3d4b95b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:00.310755 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e032ee68-ecf0-46e0-bd29-87e7e3d4b95b service placement] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:00.316272 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2468716a-3f3b-43a4-b295-0c3ee4d616b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:00.317138 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468716a-3f3b-43a4-b295-0c3ee4d616b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:00.317427 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468716a-3f3b-43a4-b295-0c3ee4d616b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:00.317711 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468716a-3f3b-43a4-b295-0c3ee4d616b3 admin admin] PATH_INFO: `/v3/projects/c1b733bd85504812bdfb51ac3c8f27eb` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:00.318890 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2468716a-3f3b-43a4-b295-0c3ee4d616b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1b733bd85504812bdfb51ac3c8f27eb)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:00.323584 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2468716a-3f3b-43a4-b295-0c3ee4d616b3 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:00.356397 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1093/2160] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:22:00 2022] PUT /identity/v3/projects/b5e107c5e832436e9a51cec433681a1d/users/0ce4f277d3234c1483a4f850548b514f/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:22:00.361424 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013e8fa-a79a-4c5b-96e2-f4072c6f82bd admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:00.361424 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-3013e8fa-a79a-4c5b-96e2-f4072c6f82bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:00.452112 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013e8fa-a79a-4c5b-96e2-f4072c6f82bd admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:00.459906 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae4108-5426-44f5-a482-c0064c84ebc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:00.460628 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae4108-5426-44f5-a482-c0064c84ebc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:00.460922 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae4108-5426-44f5-a482-c0064c84ebc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:00.461202 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae4108-5426-44f5-a482-c0064c84ebc9 admin admin] PATH_INFO: `/v3/projects/f0011eaee5b24e8aaa4e7b08263fbe04` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:00.462412 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae4108-5426-44f5-a482-c0064c84ebc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0011eaee5b24e8aaa4e7b08263fbe04)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:00.464387 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae4108-5426-44f5-a482-c0064c84ebc9 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:00.482892 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-2468716a-3f3b-43a4-b295-0c3ee4d616b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1b733bd85504812bdfb51ac3c8f27eb', 'request_id': 'req-2468716a-3f3b-43a4-b295-0c3ee4d616b3'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:00.488734 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-2468716a-3f3b-43a4-b295-0c3ee4d616b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1b733bd85504812bdfb51ac3c8f27eb', 'request_id': 'req-2468716a-3f3b-43a4-b295-0c3ee4d616b3'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:00.612049 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-72ae4108-5426-44f5-a482-c0064c84ebc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0011eaee5b24e8aaa4e7b08263fbe04', 'request_id': 'req-72ae4108-5426-44f5-a482-c0064c84ebc9'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:00.617715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-72ae4108-5426-44f5-a482-c0064c84ebc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0011eaee5b24e8aaa4e7b08263fbe04', 'request_id': 'req-72ae4108-5426-44f5-a482-c0064c84ebc9'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:00.624494 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-2468716a-3f3b-43a4-b295-0c3ee4d616b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1b733bd85504812bdfb51ac3c8f27eb', 'request_id': 'req-2468716a-3f3b-43a4-b295-0c3ee4d616b3'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:00.638151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-2468716a-3f3b-43a4-b295-0c3ee4d616b3 admin admin] The token cache is being invalidate because project c1b733bd85504812bdfb51ac3c8f27eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:22:00.638446 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-2468716a-3f3b-43a4-b295-0c3ee4d616b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:00.639108 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-2468716a-3f3b-43a4-b295-0c3ee4d616b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:00.640781 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1068/2161] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:22:00 2022] DELETE /identity/v3/projects/c1b733bd85504812bdfb51ac3c8f27eb => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:22:00.641542 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2468716a-3f3b-43a4-b295-0c3ee4d616b3 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:00.644404 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-2468716a-3f3b-43a4-b295-0c3ee4d616b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:00.713622 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2468716a-3f3b-43a4-b295-0c3ee4d616b3 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:00.740717 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-72ae4108-5426-44f5-a482-c0064c84ebc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0011eaee5b24e8aaa4e7b08263fbe04', 'request_id': 'req-72ae4108-5426-44f5-a482-c0064c84ebc9'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:00.757780 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-72ae4108-5426-44f5-a482-c0064c84ebc9 admin admin] The token cache is being invalidate because project f0011eaee5b24e8aaa4e7b08263fbe04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:22:00.758160 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-72ae4108-5426-44f5-a482-c0064c84ebc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:00.759041 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-72ae4108-5426-44f5-a482-c0064c84ebc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:00.760958 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1094/2162] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:22:00 2022] DELETE /identity/v3/projects/f0011eaee5b24e8aaa4e7b08263fbe04 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:22:00.762061 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04254e2-02ef-4b41-a4ef-8c6ba129036e None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:00.762351 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04254e2-02ef-4b41-a4ef-8c6ba129036e None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:00.764380 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04254e2-02ef-4b41-a4ef-8c6ba129036e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:00.796395 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-e04254e2-02ef-4b41-a4ef-8c6ba129036e None None] MFA Rules not processed for user `0ce4f277d3234c1483a4f850548b514f`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:00.799138 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-e04254e2-02ef-4b41-a4ef-8c6ba129036e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:00.801311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d081df1-60fd-4034-816d-34be959af554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:00.814495 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d081df1-60fd-4034-816d-34be959af554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:00.814495 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d081df1-60fd-4034-816d-34be959af554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:00.814495 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d081df1-60fd-4034-816d-34be959af554 admin admin] PATH_INFO: `/v3/users/a830bd2267f948f693c6b211293c6b89` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:00.816306 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d081df1-60fd-4034-816d-34be959af554 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a830bd2267f948f693c6b211293c6b89)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:00.818792 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d081df1-60fd-4034-816d-34be959af554 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:00.893291 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1095/2163] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:00.898447 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3da1a-f09c-4f24-86fd-89a7ac51277e None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:00.898447 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3da1a-f09c-4f24-86fd-89a7ac51277e None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:00.898447 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3da1a-f09c-4f24-86fd-89a7ac51277e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:00.917478 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-aac3da1a-f09c-4f24-86fd-89a7ac51277e None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:00.918808 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-aac3da1a-f09c-4f24-86fd-89a7ac51277e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:00.929314 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1096/2164] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:00.930345 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3da1a-f09c-4f24-86fd-89a7ac51277e None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:00.932332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-aac3da1a-f09c-4f24-86fd-89a7ac51277e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:00.946666 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3da1a-f09c-4f24-86fd-89a7ac51277e None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:00.951408 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc4ca8e-5145-48f4-8b9a-8b05c8270efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:00.952012 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4ca8e-5145-48f4-8b9a-8b05c8270efa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:00.952296 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4ca8e-5145-48f4-8b9a-8b05c8270efa admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:00.952577 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4ca8e-5145-48f4-8b9a-8b05c8270efa admin admin] PATH_INFO: `/v3/users/eeab41f9661b4c97af4aef608eaca045` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:00.953752 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc4ca8e-5145-48f4-8b9a-8b05c8270efa admin admin] RBAC: Authorizing `identity:delete_user(user_id=eeab41f9661b4c97af4aef608eaca045)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:00.955710 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc4ca8e-5145-48f4-8b9a-8b05c8270efa admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:01.014058 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-3d081df1-60fd-4034-816d-34be959af554 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a830bd2267f948f693c6b211293c6b89'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:01.014782 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-3d081df1-60fd-4034-816d-34be959af554 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a830bd2267f948f693c6b211293c6b89'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:01.015479 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-3d081df1-60fd-4034-816d-34be959af554 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a830bd2267f948f693c6b211293c6b89'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:01.022643 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-3d081df1-60fd-4034-816d-34be959af554 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a830bd2267f948f693c6b211293c6b89'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:01.035545 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-3d081df1-60fd-4034-816d-34be959af554 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a830bd2267f948f693c6b211293c6b89'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:01.130553 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1069/2165] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:22:00 2022] DELETE /identity/v3/users/a830bd2267f948f693c6b211293c6b89 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:22:01.131376 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d081df1-60fd-4034-816d-34be959af554 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:01.132743 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-3d081df1-60fd-4034-816d-34be959af554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:01.170326 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-abc4ca8e-5145-48f4-8b9a-8b05c8270efa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eeab41f9661b4c97af4aef608eaca045'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:01.178436 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-abc4ca8e-5145-48f4-8b9a-8b05c8270efa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eeab41f9661b4c97af4aef608eaca045'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:01.179214 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-abc4ca8e-5145-48f4-8b9a-8b05c8270efa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eeab41f9661b4c97af4aef608eaca045'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:01.193296 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-abc4ca8e-5145-48f4-8b9a-8b05c8270efa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eeab41f9661b4c97af4aef608eaca045'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:01.200376 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d081df1-60fd-4034-816d-34be959af554 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:01.204921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b877fa0b-d6d9-49d5-adfb-e52e2fa36988 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:01.205861 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877fa0b-d6d9-49d5-adfb-e52e2fa36988 service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:01.206147 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877fa0b-d6d9-49d5-adfb-e52e2fa36988 service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:01.206433 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877fa0b-d6d9-49d5-adfb-e52e2fa36988 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:01.211774 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-b877fa0b-d6d9-49d5-adfb-e52e2fa36988 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:01.299010 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b877fa0b-d6d9-49d5-adfb-e52e2fa36988 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:01.300602 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b877fa0b-d6d9-49d5-adfb-e52e2fa36988 service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:01.304200 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-abc4ca8e-5145-48f4-8b9a-8b05c8270efa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eeab41f9661b4c97af4aef608eaca045'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:01.311389 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1097/2166] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:22:00 2022] DELETE /identity/v3/users/eeab41f9661b4c97af4aef608eaca045 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:22:01.313190 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1070/2167] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:22:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:22:01.313594 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5084eed1-71ba-4642-9a29-1c6fd5ae14c6 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:01.313885 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5084eed1-71ba-4642-9a29-1c6fd5ae14c6 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:01.314166 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5084eed1-71ba-4642-9a29-1c6fd5ae14c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:01.335055 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-5084eed1-71ba-4642-9a29-1c6fd5ae14c6 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:01.336354 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-5084eed1-71ba-4642-9a29-1c6fd5ae14c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:01.381709 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:01.441324 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1098/2168] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:01.445685 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b877fa0b-d6d9-49d5-adfb-e52e2fa36988 service cinder] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:01.447244 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-b877fa0b-d6d9-49d5-adfb-e52e2fa36988 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:01.524229 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5084eed1-71ba-4642-9a29-1c6fd5ae14c6 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:01.535001 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-5084eed1-71ba-4642-9a29-1c6fd5ae14c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:01.549297 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5084eed1-71ba-4642-9a29-1c6fd5ae14c6 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:01.558233 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121daa91-44ee-4389-93cb-36e046325bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:01.559045 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121daa91-44ee-4389-93cb-36e046325bf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:01.559355 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121daa91-44ee-4389-93cb-36e046325bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:01.559703 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121daa91-44ee-4389-93cb-36e046325bf8 admin admin] PATH_INFO: `/v3/projects/a95d3e9a225f45a88b06bee262207b6c` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:01.562913 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121daa91-44ee-4389-93cb-36e046325bf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a95d3e9a225f45a88b06bee262207b6c)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:01.565279 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b877fa0b-d6d9-49d5-adfb-e52e2fa36988 service cinder] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:01.565682 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121daa91-44ee-4389-93cb-36e046325bf8 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:01.571118 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc63e0-270d-4eda-9d27-ae5358089633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:01.571823 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc63e0-270d-4eda-9d27-ae5358089633 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:01.572109 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc63e0-270d-4eda-9d27-ae5358089633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:01.572382 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc63e0-270d-4eda-9d27-ae5358089633 admin admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:01.573727 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bc63e0-270d-4eda-9d27-ae5358089633 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:22:01.574096 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bc63e0-270d-4eda-9d27-ae5358089633 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:01.575804 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bc63e0-270d-4eda-9d27-ae5358089633 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:01.594169 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1071/2169] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:22:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:22:01.598488 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc63e0-270d-4eda-9d27-ae5358089633 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:01.609951 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc63e0-270d-4eda-9d27-ae5358089633 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:01.614648 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6746118-a09d-4e14-9928-7bd429483dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:01.615252 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6746118-a09d-4e14-9928-7bd429483dc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:01.615569 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6746118-a09d-4e14-9928-7bd429483dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:01.615842 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6746118-a09d-4e14-9928-7bd429483dc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:01.616464 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6746118-a09d-4e14-9928-7bd429483dc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:01.618506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6746118-a09d-4e14-9928-7bd429483dc8 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:01.789164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1072/2170] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:22:01 2022] POST /identity/v3/projects => generated 385 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:22:01.794083 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6746118-a09d-4e14-9928-7bd429483dc8 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:01.799280 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-121daa91-44ee-4389-93cb-36e046325bf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a95d3e9a225f45a88b06bee262207b6c', 'request_id': 'req-121daa91-44ee-4389-93cb-36e046325bf8'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:01.805794 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-121daa91-44ee-4389-93cb-36e046325bf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a95d3e9a225f45a88b06bee262207b6c', 'request_id': 'req-121daa91-44ee-4389-93cb-36e046325bf8'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:01.884498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6746118-a09d-4e14-9928-7bd429483dc8 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:01.890124 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa2e78-7664-4f22-8366-051bbd524a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:01.890381 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa2e78-7664-4f22-8366-051bbd524a46 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:01.890969 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa2e78-7664-4f22-8366-051bbd524a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:01.891435 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa2e78-7664-4f22-8366-051bbd524a46 admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:01.892444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaa2e78-7664-4f22-8366-051bbd524a46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:01.894815 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaa2e78-7664-4f22-8366-051bbd524a46 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:01.913369 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-121daa91-44ee-4389-93cb-36e046325bf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a95d3e9a225f45a88b06bee262207b6c', 'request_id': 'req-121daa91-44ee-4389-93cb-36e046325bf8'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:01.929125 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-121daa91-44ee-4389-93cb-36e046325bf8 admin admin] The token cache is being invalidate because project a95d3e9a225f45a88b06bee262207b6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:22:01.929427 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-121daa91-44ee-4389-93cb-36e046325bf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:01.930134 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-121daa91-44ee-4389-93cb-36e046325bf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:01.932810 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1099/2171] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:22:01 2022] DELETE /identity/v3/projects/a95d3e9a225f45a88b06bee262207b6c => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:22:01.933681 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121daa91-44ee-4389-93cb-36e046325bf8 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:01.935907 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-121daa91-44ee-4389-93cb-36e046325bf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:02.026186 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:02.030611 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121daa91-44ee-4389-93cb-36e046325bf8 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:02.034609 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1073/2172] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:22:01 2022] POST /identity/v3/users => generated 293 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:22:02.036501 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835bbafc-63ee-4a85-974c-fdbc26d92498 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:02.037275 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835bbafc-63ee-4a85-974c-fdbc26d92498 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:02.037798 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835bbafc-63ee-4a85-974c-fdbc26d92498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:02.040252 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c9e64-c459-4fa6-92fa-977c24d3d062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:02.041078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c9e64-c459-4fa6-92fa-977c24d3d062 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:02.041500 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c9e64-c459-4fa6-92fa-977c24d3d062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:02.041783 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c9e64-c459-4fa6-92fa-977c24d3d062 admin admin] PATH_INFO: `/v3/projects/d896595976a448ffba8b3faa25d130a1` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:02.043384 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853c9e64-c459-4fa6-92fa-977c24d3d062 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d896595976a448ffba8b3faa25d130a1)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:02.046497 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853c9e64-c459-4fa6-92fa-977c24d3d062 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:02.065848 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-835bbafc-63ee-4a85-974c-fdbc26d92498 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:02.067470 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-835bbafc-63ee-4a85-974c-fdbc26d92498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:02.088063 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1074/2173] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:02.089068 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835bbafc-63ee-4a85-974c-fdbc26d92498 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:02.090366 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-835bbafc-63ee-4a85-974c-fdbc26d92498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:02.104058 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835bbafc-63ee-4a85-974c-fdbc26d92498 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:02.112161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047062e-a1ea-4c22-b37a-0d4ec7221c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:02.112845 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047062e-a1ea-4c22-b37a-0d4ec7221c2c admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:02.113131 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047062e-a1ea-4c22-b37a-0d4ec7221c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:02.113412 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047062e-a1ea-4c22-b37a-0d4ec7221c2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:02.114081 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047062e-a1ea-4c22-b37a-0d4ec7221c2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:02.115779 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047062e-a1ea-4c22-b37a-0d4ec7221c2c admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:02.128381 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1075/2174] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:22:02 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:22:02.129676 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c68e5-7462-4636-87eb-7b6e7e4e87e8 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:02.129966 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c68e5-7462-4636-87eb-7b6e7e4e87e8 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:02.130247 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c68e5-7462-4636-87eb-7b6e7e4e87e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:02.152368 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-b52c68e5-7462-4636-87eb-7b6e7e4e87e8 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:02.157603 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-b52c68e5-7462-4636-87eb-7b6e7e4e87e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:02.162663 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1076/2175] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:02.163320 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c68e5-7462-4636-87eb-7b6e7e4e87e8 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:02.183001 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c68e5-7462-4636-87eb-7b6e7e4e87e8 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:02.198353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-853c9e64-c459-4fa6-92fa-977c24d3d062 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd896595976a448ffba8b3faa25d130a1', 'request_id': 'req-853c9e64-c459-4fa6-92fa-977c24d3d062'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:02.207972 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-853c9e64-c459-4fa6-92fa-977c24d3d062 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd896595976a448ffba8b3faa25d130a1', 'request_id': 'req-853c9e64-c459-4fa6-92fa-977c24d3d062'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:02.264728 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36679879-271d-43ba-905b-dd0a0df5c764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:02.265951 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36679879-271d-43ba-905b-dd0a0df5c764 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:02.266233 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36679879-271d-43ba-905b-dd0a0df5c764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:02.266512 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36679879-271d-43ba-905b-dd0a0df5c764 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:02.267164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36679879-271d-43ba-905b-dd0a0df5c764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:02.269319 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36679879-271d-43ba-905b-dd0a0df5c764 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:02.282297 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1077/2176] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:22:02 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:22:02.283237 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36679879-271d-43ba-905b-dd0a0df5c764 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:02.298263 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36679879-271d-43ba-905b-dd0a0df5c764 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:02.303505 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df773cd-778c-4f7e-92ac-ddf23b241f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:02.304109 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df773cd-778c-4f7e-92ac-ddf23b241f79 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:02.304397 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df773cd-778c-4f7e-92ac-ddf23b241f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:02.304676 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df773cd-778c-4f7e-92ac-ddf23b241f79 admin admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:02.305337 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df773cd-778c-4f7e-92ac-ddf23b241f79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:22:02.305650 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df773cd-778c-4f7e-92ac-ddf23b241f79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:02.307251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df773cd-778c-4f7e-92ac-ddf23b241f79 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:02.309115 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-853c9e64-c459-4fa6-92fa-977c24d3d062 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd896595976a448ffba8b3faa25d130a1', 'request_id': 'req-853c9e64-c459-4fa6-92fa-977c24d3d062'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:02.322674 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1078/2177] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:22:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:22:02.324507 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df773cd-778c-4f7e-92ac-ddf23b241f79 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:02.325785 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-853c9e64-c459-4fa6-92fa-977c24d3d062 admin admin] The token cache is being invalidate because project d896595976a448ffba8b3faa25d130a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:22:02.325928 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-853c9e64-c459-4fa6-92fa-977c24d3d062 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:02.326743 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-853c9e64-c459-4fa6-92fa-977c24d3d062 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:02.329045 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1100/2178] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:22:01 2022] DELETE /identity/v3/projects/d896595976a448ffba8b3faa25d130a1 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:22:02.329800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c9e64-c459-4fa6-92fa-977c24d3d062 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:02.333345 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-853c9e64-c459-4fa6-92fa-977c24d3d062 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:02.348036 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c9e64-c459-4fa6-92fa-977c24d3d062 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:02.353613 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765b8ee-bcd4-4166-a7c0-252550e068b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:02.353854 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df773cd-778c-4f7e-92ac-ddf23b241f79 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:02.354523 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765b8ee-bcd4-4166-a7c0-252550e068b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:02.354893 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765b8ee-bcd4-4166-a7c0-252550e068b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:02.355253 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765b8ee-bcd4-4166-a7c0-252550e068b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:02.356034 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4765b8ee-bcd4-4166-a7c0-252550e068b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:02.358439 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4765b8ee-bcd4-4166-a7c0-252550e068b2 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:02.359948 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21af8f-6029-456b-bf7b-5d8cef6d61c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:02.360732 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21af8f-6029-456b-bf7b-5d8cef6d61c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:02.361704 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21af8f-6029-456b-bf7b-5d8cef6d61c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:02.361989 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21af8f-6029-456b-bf7b-5d8cef6d61c1 admin admin] PATH_INFO: `/v3/projects/2fb25d9e46be4c12824be2c46a5c264a/users/6078e72ec68c4763ba8814812cc16fdd/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:02.381775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b21af8f-6029-456b-bf7b-5d8cef6d61c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fb25d9e46be4c12824be2c46a5c264a, user_id=6078e72ec68c4763ba8814812cc16fdd, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:02.384814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b21af8f-6029-456b-bf7b-5d8cef6d61c1 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:02.450148 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1101/2179] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:22:02 2022] POST /identity/v3/projects => generated 387 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:22:02.451211 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23069fd9-f81e-4121-a657-7763838dca6c None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:02.451551 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23069fd9-f81e-4121-a657-7763838dca6c None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:02.451837 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23069fd9-f81e-4121-a657-7763838dca6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:02.472806 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-23069fd9-f81e-4121-a657-7763838dca6c None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:02.474038 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-23069fd9-f81e-4121-a657-7763838dca6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:02.517995 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1079/2180] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:22:02 2022] PUT /identity/v3/projects/2fb25d9e46be4c12824be2c46a5c264a/users/6078e72ec68c4763ba8814812cc16fdd/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:22:02.518846 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21af8f-6029-456b-bf7b-5d8cef6d61c1 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:02.563993 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1102/2181] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:02.563993 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951e9b6-1a59-4e74-a860-a261b448c913 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:02.563993 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951e9b6-1a59-4e74-a860-a261b448c913 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:02.563993 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951e9b6-1a59-4e74-a860-a261b448c913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:02.616337 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-6951e9b6-1a59-4e74-a860-a261b448c913 None None] MFA Rules not processed for user `6078e72ec68c4763ba8814812cc16fdd`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:02.617894 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-6951e9b6-1a59-4e74-a860-a261b448c913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:02.635034 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21af8f-6029-456b-bf7b-5d8cef6d61c1 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:02.642195 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70bdce4-6e0c-4c31-a4c8-3190b569f0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:02.643005 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70bdce4-6e0c-4c31-a4c8-3190b569f0ca admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:02.643371 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70bdce4-6e0c-4c31-a4c8-3190b569f0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:02.648934 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70bdce4-6e0c-4c31-a4c8-3190b569f0ca admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:02.648934 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70bdce4-6e0c-4c31-a4c8-3190b569f0ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:02.648934 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70bdce4-6e0c-4c31-a4c8-3190b569f0ca admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:02.726154 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1103/2182] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:02.727770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffae725-ad08-46dc-a9ac-92b3f7cd45bc None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:02.728164 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffae725-ad08-46dc-a9ac-92b3f7cd45bc None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:02.728762 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffae725-ad08-46dc-a9ac-92b3f7cd45bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:02.770185 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-dffae725-ad08-46dc-a9ac-92b3f7cd45bc None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:02.770185 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-dffae725-ad08-46dc-a9ac-92b3f7cd45bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:02.770185 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1104/2183] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:02.770185 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffae725-ad08-46dc-a9ac-92b3f7cd45bc None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:02.770185 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-dffae725-ad08-46dc-a9ac-92b3f7cd45bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:02.782170 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffae725-ad08-46dc-a9ac-92b3f7cd45bc None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:02.786953 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c8448-bf7c-4144-b952-c9ccf2bff024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:02.787621 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c8448-bf7c-4144-b952-c9ccf2bff024 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:02.787904 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c8448-bf7c-4144-b952-c9ccf2bff024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:02.788181 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c8448-bf7c-4144-b952-c9ccf2bff024 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:02.789087 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8c8448-bf7c-4144-b952-c9ccf2bff024 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:02.791682 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1080/2184] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:22:02 2022] POST /identity/v3/users => generated 294 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:22:02.791821 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8c8448-bf7c-4144-b952-c9ccf2bff024 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:02.791917 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70bdce4-6e0c-4c31-a4c8-3190b569f0ca admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:02.802618 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70bdce4-6e0c-4c31-a4c8-3190b569f0ca admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:02.807854 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43e73a-dfdf-414f-af29-a1b7fb7d6556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:02.808480 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43e73a-dfdf-414f-af29-a1b7fb7d6556 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:02.809125 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43e73a-dfdf-414f-af29-a1b7fb7d6556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:02.809410 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43e73a-dfdf-414f-af29-a1b7fb7d6556 admin admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:02.810091 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f43e73a-dfdf-414f-af29-a1b7fb7d6556 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:22:02.810397 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f43e73a-dfdf-414f-af29-a1b7fb7d6556 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:02.812607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f43e73a-dfdf-414f-af29-a1b7fb7d6556 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:02.825606 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1081/2185] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:22:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:22:02.826614 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43e73a-dfdf-414f-af29-a1b7fb7d6556 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:02.836883 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43e73a-dfdf-414f-af29-a1b7fb7d6556 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:02.841747 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df6d8f-d261-41e9-ac65-e7700b61f0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:02.842335 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df6d8f-d261-41e9-ac65-e7700b61f0cb admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:02.842685 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df6d8f-d261-41e9-ac65-e7700b61f0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:02.842958 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df6d8f-d261-41e9-ac65-e7700b61f0cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:02.843654 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79df6d8f-d261-41e9-ac65-e7700b61f0cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:02.845228 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79df6d8f-d261-41e9-ac65-e7700b61f0cb admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:02.855375 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1082/2186] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:22:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:22:02.856312 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df6d8f-d261-41e9-ac65-e7700b61f0cb admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:02.866140 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df6d8f-d261-41e9-ac65-e7700b61f0cb admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:02.871269 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0cd5e4-78ff-401e-86aa-5a2c5b87aa99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:02.871890 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0cd5e4-78ff-401e-86aa-5a2c5b87aa99 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:02.872174 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0cd5e4-78ff-401e-86aa-5a2c5b87aa99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:02.872455 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0cd5e4-78ff-401e-86aa-5a2c5b87aa99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:02.873080 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0cd5e4-78ff-401e-86aa-5a2c5b87aa99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:02.875658 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0cd5e4-78ff-401e-86aa-5a2c5b87aa99 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:02.882297 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1105/2187] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:22:02 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:22:02.882297 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c8448-bf7c-4144-b952-c9ccf2bff024 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:02.945372 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c8448-bf7c-4144-b952-c9ccf2bff024 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:02.950359 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01597d7d-c5de-48b4-9121-1104cbb02645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:02.951063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01597d7d-c5de-48b4-9121-1104cbb02645 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:02.951344 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01597d7d-c5de-48b4-9121-1104cbb02645 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:02.951723 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01597d7d-c5de-48b4-9121-1104cbb02645 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:02.952376 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01597d7d-c5de-48b4-9121-1104cbb02645 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:02.954475 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01597d7d-c5de-48b4-9121-1104cbb02645 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:02.965238 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1106/2188] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:22:02 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:22:02.966272 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01597d7d-c5de-48b4-9121-1104cbb02645 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:02.973056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1083/2189] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:22:02 2022] POST /identity/v3/projects => generated 367 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:22:02.973943 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0cd5e4-78ff-401e-86aa-5a2c5b87aa99 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:02.977848 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01597d7d-c5de-48b4-9121-1104cbb02645 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:03.036943 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f3a08b-eb80-458a-a3c7-788dd11acad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:03.037470 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3a08b-eb80-458a-a3c7-788dd11acad3 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:03.037757 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3a08b-eb80-458a-a3c7-788dd11acad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:03.037907 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0cd5e4-78ff-401e-86aa-5a2c5b87aa99 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:03.038373 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3a08b-eb80-458a-a3c7-788dd11acad3 admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:03.038651 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f3a08b-eb80-458a-a3c7-788dd11acad3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:03.041636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f3a08b-eb80-458a-a3c7-788dd11acad3 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:03.044153 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d2e1c3-d320-4883-b240-d17e50576530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:03.044153 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2e1c3-d320-4883-b240-d17e50576530 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:03.044153 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2e1c3-d320-4883-b240-d17e50576530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:03.044153 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2e1c3-d320-4883-b240-d17e50576530 admin admin] PATH_INFO: `/v3/projects/80ccf2ca9a634becb2eee93fddbd2258/users/d98949bd624a4330a473ebe5de31586a/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:03.059431 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d2e1c3-d320-4883-b240-d17e50576530 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80ccf2ca9a634becb2eee93fddbd2258, user_id=d98949bd624a4330a473ebe5de31586a, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:03.062687 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d2e1c3-d320-4883-b240-d17e50576530 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:03.140384 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1084/2190] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:22:02 2022] PUT /identity/v3/projects/80ccf2ca9a634becb2eee93fddbd2258/users/d98949bd624a4330a473ebe5de31586a/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:22:03.143438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1107/2191] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:22:02 2022] POST /identity/v3/users => generated 298 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:22:03.143592 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d2e1c3-d320-4883-b240-d17e50576530 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:03.144505 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f3a08b-eb80-458a-a3c7-788dd11acad3 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:03.154329 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f3a08b-eb80-458a-a3c7-788dd11acad3 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:03.208955 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d2e1c3-d320-4883-b240-d17e50576530 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:03.214165 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7d8868-bea0-4961-9a82-4451b23be483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:03.214777 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d8868-bea0-4961-9a82-4451b23be483 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:03.215057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d8868-bea0-4961-9a82-4451b23be483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:03.215328 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d8868-bea0-4961-9a82-4451b23be483 admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:03.215985 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7d8868-bea0-4961-9a82-4451b23be483 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:03.216096 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57690285-afd4-469f-b118-d1f7e800467f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:03.216771 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57690285-afd4-469f-b118-d1f7e800467f admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:03.216930 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57690285-afd4-469f-b118-d1f7e800467f admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:03.217204 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57690285-afd4-469f-b118-d1f7e800467f admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:03.217888 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7d8868-bea0-4961-9a82-4451b23be483 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:03.217992 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57690285-afd4-469f-b118-d1f7e800467f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:03.219578 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57690285-afd4-469f-b118-d1f7e800467f admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:03.229908 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1108/2192] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:22:03 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:22:03.231153 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e63cd-fb1a-4904-9710-77233510ab1a None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:03.231460 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e63cd-fb1a-4904-9710-77233510ab1a None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:03.231810 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e63cd-fb1a-4904-9710-77233510ab1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:03.251862 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-501e63cd-fb1a-4904-9710-77233510ab1a None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:03.253226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-501e63cd-fb1a-4904-9710-77233510ab1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:03.262141 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1109/2193] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:03.262983 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e63cd-fb1a-4904-9710-77233510ab1a None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:03.272678 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e63cd-fb1a-4904-9710-77233510ab1a None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:03.277433 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe02b3d-b736-4d31-85da-e4bf8a039069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:03.278089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe02b3d-b736-4d31-85da-e4bf8a039069 admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:03.278377 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe02b3d-b736-4d31-85da-e4bf8a039069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:03.278651 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe02b3d-b736-4d31-85da-e4bf8a039069 admin admin] PATH_INFO: `/v3/projects/40de2feaf64843c5bafd2c73dd6039a5/users/7bcd6f600e404142a98b8735d2671aa0/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:03.298217 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe02b3d-b736-4d31-85da-e4bf8a039069 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40de2feaf64843c5bafd2c73dd6039a5, user_id=7bcd6f600e404142a98b8735d2671aa0, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:03.298217 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe02b3d-b736-4d31-85da-e4bf8a039069 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:03.322354 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1085/2194] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:22:03 2022] POST /identity/v3/users => generated 284 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:22:03.322354 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7d8868-bea0-4961-9a82-4451b23be483 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:03.322354 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-5c7d8868-bea0-4961-9a82-4451b23be483 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:03.386786 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7d8868-bea0-4961-9a82-4451b23be483 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:03.390832 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b41b262-25ee-41d3-860a-2892ecc88a75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:03.391475 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b41b262-25ee-41d3-860a-2892ecc88a75 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:03.391765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b41b262-25ee-41d3-860a-2892ecc88a75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:03.392037 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b41b262-25ee-41d3-860a-2892ecc88a75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:03.401561 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b41b262-25ee-41d3-860a-2892ecc88a75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:03.403018 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b41b262-25ee-41d3-860a-2892ecc88a75 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:03.405632 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1110/2195] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:22:03 2022] PUT /identity/v3/projects/40de2feaf64843c5bafd2c73dd6039a5/users/7bcd6f600e404142a98b8735d2671aa0/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:22:03.407477 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe02b3d-b736-4d31-85da-e4bf8a039069 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:03.414386 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1086/2196] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:22:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:22:03.415227 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b41b262-25ee-41d3-860a-2892ecc88a75 service neutron] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:03.425859 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b41b262-25ee-41d3-860a-2892ecc88a75 service neutron] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:03.474966 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe02b3d-b736-4d31-85da-e4bf8a039069 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:03.480233 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775dc2f-d382-43bc-9e8a-673d8f7ff3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:03.481159 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775dc2f-d382-43bc-9e8a-673d8f7ff3f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:03.481525 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775dc2f-d382-43bc-9e8a-673d8f7ff3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:03.481874 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775dc2f-d382-43bc-9e8a-673d8f7ff3f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:03.482638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6775dc2f-d382-43bc-9e8a-673d8f7ff3f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:03.485197 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6775dc2f-d382-43bc-9e8a-673d8f7ff3f7 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:03.499731 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1111/2197] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:22:03 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:22:03.499912 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d81184-0396-4a3d-94e0-46dd05f9750b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:03.499912 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d81184-0396-4a3d-94e0-46dd05f9750b admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:03.499912 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d81184-0396-4a3d-94e0-46dd05f9750b admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:03.499912 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d81184-0396-4a3d-94e0-46dd05f9750b admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:03.500256 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d81184-0396-4a3d-94e0-46dd05f9750b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:03.500696 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775dc2f-d382-43bc-9e8a-673d8f7ff3f7 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:03.501146 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d81184-0396-4a3d-94e0-46dd05f9750b admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:03.511814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1087/2198] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:22:03 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:22:03.512136 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775dc2f-d382-43bc-9e8a-673d8f7ff3f7 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:03.522878 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6fba1a-0a60-438f-916b-5f8880f4125c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:03.523894 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6fba1a-0a60-438f-916b-5f8880f4125c admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:03.524242 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6fba1a-0a60-438f-916b-5f8880f4125c admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:03.524537 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6fba1a-0a60-438f-916b-5f8880f4125c admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:03.526748 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6fba1a-0a60-438f-916b-5f8880f4125c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:03.526899 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d81184-0396-4a3d-94e0-46dd05f9750b admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:03.528430 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6fba1a-0a60-438f-916b-5f8880f4125c admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:03.537900 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d81184-0396-4a3d-94e0-46dd05f9750b admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:03.544290 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1112/2199] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:22:03 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 12:22:03.545734 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6fba1a-0a60-438f-916b-5f8880f4125c admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:03.548502 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e65536-4d5b-489c-a88a-779fca2c649c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:03.552127 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e65536-4d5b-489c-a88a-779fca2c649c admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:03.552609 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e65536-4d5b-489c-a88a-779fca2c649c admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:03.553092 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e65536-4d5b-489c-a88a-779fca2c649c admin admin] PATH_INFO: `/v3/projects/40de2feaf64843c5bafd2c73dd6039a5/users/7bcd6f600e404142a98b8735d2671aa0/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:03.556379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e65536-4d5b-489c-a88a-779fca2c649c admin admin] RBAC: Authorizing `identity:create_grant(project_id=40de2feaf64843c5bafd2c73dd6039a5, user_id=7bcd6f600e404142a98b8735d2671aa0, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:03.560055 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6fba1a-0a60-438f-916b-5f8880f4125c admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:03.560401 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e65536-4d5b-489c-a88a-779fca2c649c admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:03.567942 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c39b3f-be1e-4eca-96de-3e4c62302c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:03.568731 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c39b3f-be1e-4eca-96de-3e4c62302c2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:03.569117 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c39b3f-be1e-4eca-96de-3e4c62302c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:03.569499 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c39b3f-be1e-4eca-96de-3e4c62302c2c admin admin] PATH_INFO: `/v3/projects/a15ccd503f48412c866cf7894ba2bffe/users/87cd773990d74f7e952b0176b07694a6/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:03.586985 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c39b3f-be1e-4eca-96de-3e4c62302c2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a15ccd503f48412c866cf7894ba2bffe, user_id=87cd773990d74f7e952b0176b07694a6, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:03.595467 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c39b3f-be1e-4eca-96de-3e4c62302c2c admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:03.595647 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1088/2200] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:22:03 2022] PUT /identity/v3/projects/40de2feaf64843c5bafd2c73dd6039a5/users/7bcd6f600e404142a98b8735d2671aa0/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:22:03.603017 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b6011-b579-4282-8456-ef1ee3e70677 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:03.603364 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b6011-b579-4282-8456-ef1ee3e70677 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:03.603698 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b6011-b579-4282-8456-ef1ee3e70677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:03.636111 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-880b6011-b579-4282-8456-ef1ee3e70677 None None] MFA Rules not processed for user `7bcd6f600e404142a98b8735d2671aa0`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:03.637402 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-880b6011-b579-4282-8456-ef1ee3e70677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:03.641837 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:03.709121 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1113/2201] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:22:03 2022] PUT /identity/v3/projects/a15ccd503f48412c866cf7894ba2bffe/users/87cd773990d74f7e952b0176b07694a6/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:22:03.712286 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed85e2-830d-41b7-a5b9-72c2dcaf6c20 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:03.712286 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed85e2-830d-41b7-a5b9-72c2dcaf6c20 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:03.712286 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed85e2-830d-41b7-a5b9-72c2dcaf6c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:03.715953 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1089/2202] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:03 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:22:03.727965 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b6011-b579-4282-8456-ef1ee3e70677 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:03.728782 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-880b6011-b579-4282-8456-ef1ee3e70677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:03.758666 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-70ed85e2-830d-41b7-a5b9-72c2dcaf6c20 None None] MFA Rules not processed for user `87cd773990d74f7e952b0176b07694a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:03.758666 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-70ed85e2-830d-41b7-a5b9-72c2dcaf6c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:03.857432 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b6011-b579-4282-8456-ef1ee3e70677 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:03.880994 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1114/2203] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:03.882625 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ed85e2-830d-41b7-a5b9-72c2dcaf6c20 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:03.912376 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3c46ac-f37b-46c4-ae7e-e1c5ebd2d01c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:03.913277 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c46ac-f37b-46c4-ae7e-e1c5ebd2d01c service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:03.953765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c46ac-f37b-46c4-ae7e-e1c5ebd2d01c service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:03.953765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c46ac-f37b-46c4-ae7e-e1c5ebd2d01c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:03.985688 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ed85e2-830d-41b7-a5b9-72c2dcaf6c20 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:03.992298 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d17975-6a90-4df5-9434-ea20c376ad69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:04.020288 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d17975-6a90-4df5-9434-ea20c376ad69 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:04.020288 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d17975-6a90-4df5-9434-ea20c376ad69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:04.020288 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d17975-6a90-4df5-9434-ea20c376ad69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:04.020288 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d17975-6a90-4df5-9434-ea20c376ad69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:04.020288 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d17975-6a90-4df5-9434-ea20c376ad69 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:04.021063 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3c46ac-f37b-46c4-ae7e-e1c5ebd2d01c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:04.021063 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3c46ac-f37b-46c4-ae7e-e1c5ebd2d01c service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:04.021063 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1090/2204] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:22:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:22:04.198560 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1115/2205] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:22:03 2022] POST /identity/v3/projects => generated 367 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:22:04.201679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3c46ac-f37b-46c4-ae7e-e1c5ebd2d01c service cinder] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:04.216280 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3c46ac-f37b-46c4-ae7e-e1c5ebd2d01c service cinder] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:04.277998 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461443f-7b67-4fb6-be77-522e00461b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:04.278606 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461443f-7b67-4fb6-be77-522e00461b75 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:04.278891 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461443f-7b67-4fb6-be77-522e00461b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:04.279161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461443f-7b67-4fb6-be77-522e00461b75 admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:04.279845 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6461443f-7b67-4fb6-be77-522e00461b75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:04.281890 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6461443f-7b67-4fb6-be77-522e00461b75 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:04.379774 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:04.387045 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1091/2206] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:22:04 2022] POST /identity/v3/users => generated 290 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:22:04.391726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d17975-6a90-4df5-9434-ea20c376ad69 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:04.402567 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d17975-6a90-4df5-9434-ea20c376ad69 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:04.408764 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32a986-1bab-4af9-8ba9-f20e50ef86ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:04.409386 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32a986-1bab-4af9-8ba9-f20e50ef86ac admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:04.409670 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32a986-1bab-4af9-8ba9-f20e50ef86ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:04.410095 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32a986-1bab-4af9-8ba9-f20e50ef86ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:04.410811 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b32a986-1bab-4af9-8ba9-f20e50ef86ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:04.412479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b32a986-1bab-4af9-8ba9-f20e50ef86ac admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:04.425496 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1116/2207] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:22:04.429746 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461443f-7b67-4fb6-be77-522e00461b75 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:04.440719 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461443f-7b67-4fb6-be77-522e00461b75 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:04.447875 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3c8e0-e56d-4903-8f37-4ddebe63f773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:04.448547 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3c8e0-e56d-4903-8f37-4ddebe63f773 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:04.448897 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3c8e0-e56d-4903-8f37-4ddebe63f773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:04.449116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3c8e0-e56d-4903-8f37-4ddebe63f773 admin admin] PATH_INFO: `/v3/projects/e96813b43cff41eb995642099477b22d/users/04580970d0854afa913969c32dcc787c/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:04.469319 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f3c8e0-e56d-4903-8f37-4ddebe63f773 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e96813b43cff41eb995642099477b22d, user_id=04580970d0854afa913969c32dcc787c, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:04.472812 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f3c8e0-e56d-4903-8f37-4ddebe63f773 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:04.632257 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1092/2208] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:22:04 2022] PUT /identity/v3/projects/e96813b43cff41eb995642099477b22d/users/04580970d0854afa913969c32dcc787c/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:22:04.636428 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32a986-1bab-4af9-8ba9-f20e50ef86ac admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:04.649708 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32a986-1bab-4af9-8ba9-f20e50ef86ac admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:04.723461 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2f812-e338-4658-a75f-e9e08cb26f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:04.724429 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2f812-e338-4658-a75f-e9e08cb26f4a admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:04.724777 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2f812-e338-4658-a75f-e9e08cb26f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:04.725063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2f812-e338-4658-a75f-e9e08cb26f4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:04.725721 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b2f812-e338-4658-a75f-e9e08cb26f4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:04.727442 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b2f812-e338-4658-a75f-e9e08cb26f4a admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:04.741707 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1117/2209] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:22:04.744307 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3c8e0-e56d-4903-8f37-4ddebe63f773 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:04.763888 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3c8e0-e56d-4903-8f37-4ddebe63f773 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:04.769850 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237857b4-1374-40f5-808b-0478b3feada3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:04.771131 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237857b4-1374-40f5-808b-0478b3feada3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:04.771598 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237857b4-1374-40f5-808b-0478b3feada3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:04.772203 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237857b4-1374-40f5-808b-0478b3feada3 admin admin] PATH_INFO: `/v3/projects/e96813b43cff41eb995642099477b22d/users/04580970d0854afa913969c32dcc787c/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:04.774960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237857b4-1374-40f5-808b-0478b3feada3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e96813b43cff41eb995642099477b22d, user_id=04580970d0854afa913969c32dcc787c, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:04.778328 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237857b4-1374-40f5-808b-0478b3feada3 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:04.779551 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2f812-e338-4658-a75f-e9e08cb26f4a admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:04.782510 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-33b2f812-e338-4658-a75f-e9e08cb26f4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:04.797555 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2f812-e338-4658-a75f-e9e08cb26f4a admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:04.800277 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1093/2210] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:22:04 2022] PUT /identity/v3/projects/e96813b43cff41eb995642099477b22d/users/04580970d0854afa913969c32dcc787c/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:22:04.804607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b6398-4786-4b78-9d13-96a3d859baba None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:04.804941 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b6398-4786-4b78-9d13-96a3d859baba None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:04.805175 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b6398-4786-4b78-9d13-96a3d859baba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:04.836586 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-eb2b6398-4786-4b78-9d13-96a3d859baba None None] MFA Rules not processed for user `04580970d0854afa913969c32dcc787c`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:04.843224 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-eb2b6398-4786-4b78-9d13-96a3d859baba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:04.882432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1353353d-2882-4b09-ae2a-1da0e98c4e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:04.882991 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353353d-2882-4b09-ae2a-1da0e98c4e17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:04.883278 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353353d-2882-4b09-ae2a-1da0e98c4e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:04.883617 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353353d-2882-4b09-ae2a-1da0e98c4e17 admin admin] PATH_INFO: `/v3/users/0ce4f277d3234c1483a4f850548b514f` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:04.884885 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1353353d-2882-4b09-ae2a-1da0e98c4e17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ce4f277d3234c1483a4f850548b514f)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:04.888045 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1353353d-2882-4b09-ae2a-1da0e98c4e17 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:04.919658 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1094/2211] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:04 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:04.930798 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b6398-4786-4b78-9d13-96a3d859baba None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:04.998022 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b6398-4786-4b78-9d13-96a3d859baba None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:05.005174 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08247d2d-9e5a-4bbb-8231-3d38c7c279ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:05.005902 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08247d2d-9e5a-4bbb-8231-3d38c7c279ce service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:05.006284 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08247d2d-9e5a-4bbb-8231-3d38c7c279ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:05.006678 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08247d2d-9e5a-4bbb-8231-3d38c7c279ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:05.018448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08247d2d-9e5a-4bbb-8231-3d38c7c279ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:05.020215 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08247d2d-9e5a-4bbb-8231-3d38c7c279ce service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:05.031049 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1095/2212] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:22:04 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:22:05.189785 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-1353353d-2882-4b09-ae2a-1da0e98c4e17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ce4f277d3234c1483a4f850548b514f'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:05.201999 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-1353353d-2882-4b09-ae2a-1da0e98c4e17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ce4f277d3234c1483a4f850548b514f'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:05.202804 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-1353353d-2882-4b09-ae2a-1da0e98c4e17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ce4f277d3234c1483a4f850548b514f'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:05.218781 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-1353353d-2882-4b09-ae2a-1da0e98c4e17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ce4f277d3234c1483a4f850548b514f'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:05.363281 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-1353353d-2882-4b09-ae2a-1da0e98c4e17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ce4f277d3234c1483a4f850548b514f'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:05.373122 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1118/2213] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:22:04 2022] DELETE /identity/v3/users/0ce4f277d3234c1483a4f850548b514f => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:22:05.379060 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6800bd-b70a-4efd-a8b1-4603fe0d9d17 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:05.383754 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6800bd-b70a-4efd-a8b1-4603fe0d9d17 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:05.384178 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6800bd-b70a-4efd-a8b1-4603fe0d9d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:05.409249 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-8d6800bd-b70a-4efd-a8b1-4603fe0d9d17 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:05.410776 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-8d6800bd-b70a-4efd-a8b1-4603fe0d9d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:05.483282 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1096/2214] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:22:05.497937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1353353d-2882-4b09-ae2a-1da0e98c4e17 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:05.500005 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-1353353d-2882-4b09-ae2a-1da0e98c4e17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:05.575319 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1353353d-2882-4b09-ae2a-1da0e98c4e17 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:05.584693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ae91a-5bab-4e07-b308-dabb1e4a5d79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:05.585707 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ae91a-5bab-4e07-b308-dabb1e4a5d79 service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:05.585997 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ae91a-5bab-4e07-b308-dabb1e4a5d79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:05.586276 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ae91a-5bab-4e07-b308-dabb1e4a5d79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:05.597127 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ae91a-5bab-4e07-b308-dabb1e4a5d79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:05.598831 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ae91a-5bab-4e07-b308-dabb1e4a5d79 service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:05.610228 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1119/2215] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:22:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:22:06.350614 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6800bd-b70a-4efd-a8b1-4603fe0d9d17 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:06.350614 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-8d6800bd-b70a-4efd-a8b1-4603fe0d9d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:06.365691 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6800bd-b70a-4efd-a8b1-4603fe0d9d17 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:06.386271 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9210deaa-de02-4f02-9748-6436b2a1b323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:06.386271 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210deaa-de02-4f02-9748-6436b2a1b323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:06.386271 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210deaa-de02-4f02-9748-6436b2a1b323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:06.386271 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210deaa-de02-4f02-9748-6436b2a1b323 admin admin] PATH_INFO: `/v3/projects/b5e107c5e832436e9a51cec433681a1d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:06.387153 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9210deaa-de02-4f02-9748-6436b2a1b323 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5e107c5e832436e9a51cec433681a1d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:06.393412 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9210deaa-de02-4f02-9748-6436b2a1b323 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:06.644175 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-9210deaa-de02-4f02-9748-6436b2a1b323 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5e107c5e832436e9a51cec433681a1d', 'request_id': 'req-9210deaa-de02-4f02-9748-6436b2a1b323'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:06.644308 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:06.654464 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-9210deaa-de02-4f02-9748-6436b2a1b323 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5e107c5e832436e9a51cec433681a1d', 'request_id': 'req-9210deaa-de02-4f02-9748-6436b2a1b323'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:06.761124 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-9210deaa-de02-4f02-9748-6436b2a1b323 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5e107c5e832436e9a51cec433681a1d', 'request_id': 'req-9210deaa-de02-4f02-9748-6436b2a1b323'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:06.815310 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-9210deaa-de02-4f02-9748-6436b2a1b323 admin admin] The token cache is being invalidate because project b5e107c5e832436e9a51cec433681a1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:22:06.815845 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-9210deaa-de02-4f02-9748-6436b2a1b323 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:06.817679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-9210deaa-de02-4f02-9748-6436b2a1b323 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:06.823903 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1097/2216] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:22:06 2022] DELETE /identity/v3/projects/b5e107c5e832436e9a51cec433681a1d => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:22:06.828798 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565e825-3e96-43ab-a8f5-a59b611af67a None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:06.831640 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565e825-3e96-43ab-a8f5-a59b611af67a None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:06.838957 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565e825-3e96-43ab-a8f5-a59b611af67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:06.868426 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-e565e825-3e96-43ab-a8f5-a59b611af67a None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:06.869884 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-e565e825-3e96-43ab-a8f5-a59b611af67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:07.006132 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1120/2217] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:07.014492 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b7269-8fe0-4db3-a596-236016703d54 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:07.015750 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b7269-8fe0-4db3-a596-236016703d54 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:07.016328 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b7269-8fe0-4db3-a596-236016703d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:07.081297 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-fe1b7269-8fe0-4db3-a596-236016703d54 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:07.085981 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-fe1b7269-8fe0-4db3-a596-236016703d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:07.098396 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:07.117169 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1098/2218] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:07.121889 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e565e825-3e96-43ab-a8f5-a59b611af67a None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:07.151788 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e565e825-3e96-43ab-a8f5-a59b611af67a None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:07.168194 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6dc59d-d04c-4c80-b690-7b503c9284bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:07.168998 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6dc59d-d04c-4c80-b690-7b503c9284bb admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:07.169252 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6dc59d-d04c-4c80-b690-7b503c9284bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:07.169529 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6dc59d-d04c-4c80-b690-7b503c9284bb admin admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:07.170229 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6dc59d-d04c-4c80-b690-7b503c9284bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:22:07.170524 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6dc59d-d04c-4c80-b690-7b503c9284bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:07.172214 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6dc59d-d04c-4c80-b690-7b503c9284bb admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:07.203099 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1121/2219] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:22:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:22:07.206494 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1b7269-8fe0-4db3-a596-236016703d54 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:07.216615 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1b7269-8fe0-4db3-a596-236016703d54 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:07.223192 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fcd517-4ae2-4fdc-bea1-546d38ee1d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:07.224015 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fcd517-4ae2-4fdc-bea1-546d38ee1d97 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:07.224400 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fcd517-4ae2-4fdc-bea1-546d38ee1d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:07.224739 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fcd517-4ae2-4fdc-bea1-546d38ee1d97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:07.225642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fcd517-4ae2-4fdc-bea1-546d38ee1d97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:07.228643 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fcd517-4ae2-4fdc-bea1-546d38ee1d97 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:07.367976 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1099/2220] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:22:07 2022] POST /identity/v3/projects => generated 369 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:22:07.372726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6dc59d-d04c-4c80-b690-7b503c9284bb admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:07.397671 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6dc59d-d04c-4c80-b690-7b503c9284bb admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:07.493946 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed29384-611d-43a7-94d6-e20acd717f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:07.496193 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed29384-611d-43a7-94d6-e20acd717f33 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:07.504045 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed29384-611d-43a7-94d6-e20acd717f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:07.504045 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed29384-611d-43a7-94d6-e20acd717f33 admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:07.504045 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed29384-611d-43a7-94d6-e20acd717f33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:07.504045 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed29384-611d-43a7-94d6-e20acd717f33 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:07.639678 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1122/2221] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:22:07 2022] POST /identity/v3/users => generated 285 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:22:07.644092 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fcd517-4ae2-4fdc-bea1-546d38ee1d97 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:07.660417 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fcd517-4ae2-4fdc-bea1-546d38ee1d97 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:07.661284 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c2d97-8926-41ce-86af-11319a5d4abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:07.662109 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c2d97-8926-41ce-86af-11319a5d4abd admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:07.662395 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c2d97-8926-41ce-86af-11319a5d4abd admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:07.662676 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c2d97-8926-41ce-86af-11319a5d4abd admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:07.663490 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c2d97-8926-41ce-86af-11319a5d4abd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:07.664947 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c2d97-8926-41ce-86af-11319a5d4abd admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:07.683813 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1100/2222] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:22:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:22:07.684160 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed29384-611d-43a7-94d6-e20acd717f33 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:07.699205 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed29384-611d-43a7-94d6-e20acd717f33 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:07.706323 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea98d76-6654-4921-b42d-be55ce4606f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:07.706323 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea98d76-6654-4921-b42d-be55ce4606f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:07.706323 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea98d76-6654-4921-b42d-be55ce4606f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:07.706323 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea98d76-6654-4921-b42d-be55ce4606f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:07.706858 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea98d76-6654-4921-b42d-be55ce4606f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:07.712751 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea98d76-6654-4921-b42d-be55ce4606f5 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:07.748990 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1123/2223] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:22:07 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:22:07.751658 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c2d97-8926-41ce-86af-11319a5d4abd admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:07.765678 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c2d97-8926-41ce-86af-11319a5d4abd admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:07.780031 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bcb355-549a-4504-bc3a-d6d284976039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:07.784410 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bcb355-549a-4504-bc3a-d6d284976039 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:07.784410 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bcb355-549a-4504-bc3a-d6d284976039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:07.784410 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bcb355-549a-4504-bc3a-d6d284976039 admin admin] PATH_INFO: `/v3/projects/53547de8e0b14177859608dda3893780/users/42c92a39336b4127876efdef9b1e1763/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:07.807271 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bcb355-549a-4504-bc3a-d6d284976039 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53547de8e0b14177859608dda3893780, user_id=42c92a39336b4127876efdef9b1e1763, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:07.811056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bcb355-549a-4504-bc3a-d6d284976039 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:07.923709 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1101/2224] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:22:07 2022] PUT /identity/v3/projects/53547de8e0b14177859608dda3893780/users/42c92a39336b4127876efdef9b1e1763/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:22:07.934063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467421a-cb24-415d-86a7-e822f30ff383 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:07.934365 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467421a-cb24-415d-86a7-e822f30ff383 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:07.934653 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467421a-cb24-415d-86a7-e822f30ff383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:07.971155 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-d467421a-cb24-415d-86a7-e822f30ff383 None None] MFA Rules not processed for user `42c92a39336b4127876efdef9b1e1763`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:07.972754 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-d467421a-cb24-415d-86a7-e822f30ff383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:08.082461 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1124/2225] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:08.106460 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bcb355-549a-4504-bc3a-d6d284976039 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:08.108678 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-57bcb355-549a-4504-bc3a-d6d284976039 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:08.214585 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bcb355-549a-4504-bc3a-d6d284976039 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:08.232929 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365001a7-2fcb-4725-9b7c-e76ce6ea98ab service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd020063702b0481e9f4111680766f7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:08.235354 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365001a7-2fcb-4725-9b7c-e76ce6ea98ab service glance] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:08.235739 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365001a7-2fcb-4725-9b7c-e76ce6ea98ab service glance] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:08.236128 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365001a7-2fcb-4725-9b7c-e76ce6ea98ab service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:08.263605 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365001a7-2fcb-4725-9b7c-e76ce6ea98ab service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:08.266395 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365001a7-2fcb-4725-9b7c-e76ce6ea98ab service glance] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:08.278044 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1102/2226] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:22:08 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:22:08.490819 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97892b94-e2a2-4265-bc50-ee2f669fca50 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:08.491533 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97892b94-e2a2-4265-bc50-ee2f669fca50 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:08.492104 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97892b94-e2a2-4265-bc50-ee2f669fca50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:08.534000 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-97892b94-e2a2-4265-bc50-ee2f669fca50 None None] MFA Rules not processed for user `b6880278f7304051b9ef03e927637e74`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:08.536307 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-97892b94-e2a2-4265-bc50-ee2f669fca50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:08.660450 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1125/2227] 10.222.0.98 () {58 vars in 1009 bytes} [Thu Aug 18 12:22:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:08.725888 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365001a7-2fcb-4725-9b7c-e76ce6ea98ab service glance] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:08.727505 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-365001a7-2fcb-4725-9b7c-e76ce6ea98ab service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:08.822665 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365001a7-2fcb-4725-9b7c-e76ce6ea98ab service glance] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:08.830890 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458fb12a-88aa-48fa-b5ce-1eb6ff9171f2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771591fe31cb4e8c9c4320c2a7b2719d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:08.831624 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458fb12a-88aa-48fa-b5ce-1eb6ff9171f2 service swift] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:08.831909 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458fb12a-88aa-48fa-b5ce-1eb6ff9171f2 service swift] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:08.832183 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458fb12a-88aa-48fa-b5ce-1eb6ff9171f2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:08.853295 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458fb12a-88aa-48fa-b5ce-1eb6ff9171f2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:08.855015 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458fb12a-88aa-48fa-b5ce-1eb6ff9171f2 service swift] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:08.865667 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1103/2228] 10.222.0.98 () {60 vars in 1384 bytes} [Thu Aug 18 12:22:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 12:22:09.258828 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:09.818503 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:10.377445 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7190c-ec0b-46db-a82b-71d1f1617d6c None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:10.378509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7190c-ec0b-46db-a82b-71d1f1617d6c None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:10.379003 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7190c-ec0b-46db-a82b-71d1f1617d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:10.425018 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458fb12a-88aa-48fa-b5ce-1eb6ff9171f2 service swift] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:10.426688 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-458fb12a-88aa-48fa-b5ce-1eb6ff9171f2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:10.430622 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-a9d7190c-ec0b-46db-a82b-71d1f1617d6c None None] MFA Rules not processed for user `b6880278f7304051b9ef03e927637e74`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:10.433138 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-a9d7190c-ec0b-46db-a82b-71d1f1617d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:10.443358 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1126/2229] 10.222.0.98 () {58 vars in 967 bytes} [Thu Aug 18 12:22:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:10.473291 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d7190c-ec0b-46db-a82b-71d1f1617d6c None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:10.504220 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d7190c-ec0b-46db-a82b-71d1f1617d6c None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:10.532319 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d129f1-f936-4866-8432-fd22bded90fb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771591fe31cb4e8c9c4320c2a7b2719d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:10.533371 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d129f1-f936-4866-8432-fd22bded90fb service swift] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:10.533826 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d129f1-f936-4866-8432-fd22bded90fb service swift] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:10.534268 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d129f1-f936-4866-8432-fd22bded90fb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:10.561142 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d129f1-f936-4866-8432-fd22bded90fb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:10.567794 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d129f1-f936-4866-8432-fd22bded90fb service swift] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:10.582193 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458fb12a-88aa-48fa-b5ce-1eb6ff9171f2 service swift] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:10.597953 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1127/2230] 10.222.0.98 () {60 vars in 1384 bytes} [Thu Aug 18 12:22:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 12:22:10.606406 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d129f1-f936-4866-8432-fd22bded90fb service swift] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:10.607196 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a98d0-9dbb-4b95-9a99-11fb71b87086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:10.608323 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-95d129f1-f936-4866-8432-fd22bded90fb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:10.609028 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a98d0-9dbb-4b95-9a99-11fb71b87086 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:10.612534 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a98d0-9dbb-4b95-9a99-11fb71b87086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:10.614671 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a98d0-9dbb-4b95-9a99-11fb71b87086 admin admin] PATH_INFO: `/v3/users/87cd773990d74f7e952b0176b07694a6` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:10.625244 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9a98d0-9dbb-4b95-9a99-11fb71b87086 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87cd773990d74f7e952b0176b07694a6)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:10.627602 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9a98d0-9dbb-4b95-9a99-11fb71b87086 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:10.728812 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d129f1-f936-4866-8432-fd22bded90fb service swift] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:10.735616 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd6374-db8a-422d-9def-af715035c696 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:10.738827 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd6374-db8a-422d-9def-af715035c696 service cinder] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:10.739130 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd6374-db8a-422d-9def-af715035c696 service cinder] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:10.739443 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd6374-db8a-422d-9def-af715035c696 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:10.742769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-fddd6374-db8a-422d-9def-af715035c696 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:10.867209 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd6374-db8a-422d-9def-af715035c696 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:10.868969 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd6374-db8a-422d-9def-af715035c696 service cinder] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:10.879532 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1128/2231] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:22:10 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:22:10.948327 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-1e9a98d0-9dbb-4b95-9a99-11fb71b87086 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87cd773990d74f7e952b0176b07694a6'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:10.949173 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-1e9a98d0-9dbb-4b95-9a99-11fb71b87086 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87cd773990d74f7e952b0176b07694a6'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:10.949929 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-1e9a98d0-9dbb-4b95-9a99-11fb71b87086 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87cd773990d74f7e952b0176b07694a6'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:10.960277 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-1e9a98d0-9dbb-4b95-9a99-11fb71b87086 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87cd773990d74f7e952b0176b07694a6'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:10.991016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-1e9a98d0-9dbb-4b95-9a99-11fb71b87086 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87cd773990d74f7e952b0176b07694a6'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:11.078340 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1104/2232] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:22:10 2022] DELETE /identity/v3/users/87cd773990d74f7e952b0176b07694a6 => generated 0 bytes in 652 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:22:11.082783 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618e4b6-eb29-4525-9723-73fc3b9e6f36 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:11.083063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618e4b6-eb29-4525-9723-73fc3b9e6f36 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:11.083511 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618e4b6-eb29-4525-9723-73fc3b9e6f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:11.132881 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-0618e4b6-eb29-4525-9723-73fc3b9e6f36 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:11.138403 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-0618e4b6-eb29-4525-9723-73fc3b9e6f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:11.270343 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1129/2233] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:11.281861 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a98d0-9dbb-4b95-9a99-11fb71b87086 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:11.283785 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-1e9a98d0-9dbb-4b95-9a99-11fb71b87086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:11.391097 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a98d0-9dbb-4b95-9a99-11fb71b87086 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:11.395249 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9539ac-b51f-4ddf-bb24-f4526ae57fa1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:11.396222 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9539ac-b51f-4ddf-bb24-f4526ae57fa1 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:11.396662 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9539ac-b51f-4ddf-bb24-f4526ae57fa1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:11.397175 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9539ac-b51f-4ddf-bb24-f4526ae57fa1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:11.409431 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9539ac-b51f-4ddf-bb24-f4526ae57fa1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:11.409933 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9539ac-b51f-4ddf-bb24-f4526ae57fa1 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:11.423111 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1105/2234] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:22:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:22:11.516103 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:11.581379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b50c7-ba18-4ccd-8b9d-37309ec4c8bf None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:11.583782 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b50c7-ba18-4ccd-8b9d-37309ec4c8bf None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:11.583912 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b50c7-ba18-4ccd-8b9d-37309ec4c8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:11.627738 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-401b50c7-ba18-4ccd-8b9d-37309ec4c8bf None None] MFA Rules not processed for user `d98949bd624a4330a473ebe5de31586a`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:11.629456 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-401b50c7-ba18-4ccd-8b9d-37309ec4c8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:11.715612 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23dc5e6-4cad-4093-af4b-03a98f754615 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:11.716242 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23dc5e6-4cad-4093-af4b-03a98f754615 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:11.716701 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23dc5e6-4cad-4093-af4b-03a98f754615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:11.734358 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1130/2235] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:11 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:11.739016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-e23dc5e6-4cad-4093-af4b-03a98f754615 None None] MFA Rules not processed for user `b6880278f7304051b9ef03e927637e74`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:11.740729 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-e23dc5e6-4cad-4093-af4b-03a98f754615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:11.751399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:11.766306 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b50c7-ba18-4ccd-8b9d-37309ec4c8bf None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:11.773612 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-401b50c7-ba18-4ccd-8b9d-37309ec4c8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:11.854964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1106/2236] 10.222.0.98 () {58 vars in 1009 bytes} [Thu Aug 18 12:22:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:22:11.868869 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23dc5e6-4cad-4093-af4b-03a98f754615 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:11.873123 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-e23dc5e6-4cad-4093-af4b-03a98f754615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:11.972166 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b50c7-ba18-4ccd-8b9d-37309ec4c8bf None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:11.992733 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c6b73e-b7a0-40df-9d9c-5e9fdfa13803 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:11.995894 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c6b73e-b7a0-40df-9d9c-5e9fdfa13803 service nova] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:12.000656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c6b73e-b7a0-40df-9d9c-5e9fdfa13803 service nova] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:12.000656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c6b73e-b7a0-40df-9d9c-5e9fdfa13803 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:12.015787 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c6b73e-b7a0-40df-9d9c-5e9fdfa13803 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:12.017480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c6b73e-b7a0-40df-9d9c-5e9fdfa13803 service nova] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:12.030176 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1131/2237] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:22:11 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:22:12.031156 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23dc5e6-4cad-4093-af4b-03a98f754615 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:12.037689 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcad757-ef96-4335-a9e1-150fe1fc587a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771591fe31cb4e8c9c4320c2a7b2719d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:12.038504 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcad757-ef96-4335-a9e1-150fe1fc587a service swift] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:12.038681 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcad757-ef96-4335-a9e1-150fe1fc587a service swift] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:12.038958 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcad757-ef96-4335-a9e1-150fe1fc587a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:12.050846 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcad757-ef96-4335-a9e1-150fe1fc587a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:12.055987 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcad757-ef96-4335-a9e1-150fe1fc587a service swift] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:12.069724 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1107/2238] 10.222.0.98 () {60 vars in 1384 bytes} [Thu Aug 18 12:22:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 12:22:12.259964 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c6b73e-b7a0-40df-9d9c-5e9fdfa13803 service nova] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:12.261750 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-67c6b73e-b7a0-40df-9d9c-5e9fdfa13803 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:12.277022 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c6b73e-b7a0-40df-9d9c-5e9fdfa13803 service nova] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:12.284566 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926ce4b-d8f2-49ed-8cbe-0df8b25616fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:12.285367 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926ce4b-d8f2-49ed-8cbe-0df8b25616fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:12.288864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926ce4b-d8f2-49ed-8cbe-0df8b25616fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:12.288864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926ce4b-d8f2-49ed-8cbe-0df8b25616fa admin admin] PATH_INFO: `/v3/projects/a15ccd503f48412c866cf7894ba2bffe` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:12.291482 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e926ce4b-d8f2-49ed-8cbe-0df8b25616fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=a15ccd503f48412c866cf7894ba2bffe)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:12.293674 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e926ce4b-d8f2-49ed-8cbe-0df8b25616fa admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:12.479130 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-e926ce4b-d8f2-49ed-8cbe-0df8b25616fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a15ccd503f48412c866cf7894ba2bffe', 'request_id': 'req-e926ce4b-d8f2-49ed-8cbe-0df8b25616fa'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:12.486152 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-e926ce4b-d8f2-49ed-8cbe-0df8b25616fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a15ccd503f48412c866cf7894ba2bffe', 'request_id': 'req-e926ce4b-d8f2-49ed-8cbe-0df8b25616fa'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:12.604242 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-e926ce4b-d8f2-49ed-8cbe-0df8b25616fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a15ccd503f48412c866cf7894ba2bffe', 'request_id': 'req-e926ce4b-d8f2-49ed-8cbe-0df8b25616fa'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:12.617925 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-e926ce4b-d8f2-49ed-8cbe-0df8b25616fa admin admin] The token cache is being invalidate because project a15ccd503f48412c866cf7894ba2bffe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:22:12.618153 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-e926ce4b-d8f2-49ed-8cbe-0df8b25616fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:12.619768 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-e926ce4b-d8f2-49ed-8cbe-0df8b25616fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:12.621664 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1132/2239] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:22:12 2022] DELETE /identity/v3/projects/a15ccd503f48412c866cf7894ba2bffe => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:22:12.625578 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcad757-ef96-4335-a9e1-150fe1fc587a service swift] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:12.639283 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcad757-ef96-4335-a9e1-150fe1fc587a service swift] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:12.705055 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88672c14-ecd8-49a6-a3fd-b14fcf677029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:12.705730 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88672c14-ecd8-49a6-a3fd-b14fcf677029 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:12.706014 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88672c14-ecd8-49a6-a3fd-b14fcf677029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:12.706290 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88672c14-ecd8-49a6-a3fd-b14fcf677029 admin admin] PATH_INFO: `/v3/users/04580970d0854afa913969c32dcc787c` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:12.707705 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88672c14-ecd8-49a6-a3fd-b14fcf677029 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04580970d0854afa913969c32dcc787c)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:12.712390 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88672c14-ecd8-49a6-a3fd-b14fcf677029 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:12.827895 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8552d8-3b55-4e43-ab91-8758e85bc9bf None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:12.828270 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8552d8-3b55-4e43-ab91-8758e85bc9bf None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:12.828471 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8552d8-3b55-4e43-ab91-8758e85bc9bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:12.868895 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-2a8552d8-3b55-4e43-ab91-8758e85bc9bf None None] MFA Rules not processed for user `b6880278f7304051b9ef03e927637e74`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:12.870088 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-2a8552d8-3b55-4e43-ab91-8758e85bc9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:12.920036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-88672c14-ecd8-49a6-a3fd-b14fcf677029 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04580970d0854afa913969c32dcc787c'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:12.921847 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-88672c14-ecd8-49a6-a3fd-b14fcf677029 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04580970d0854afa913969c32dcc787c'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:12.923615 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-88672c14-ecd8-49a6-a3fd-b14fcf677029 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04580970d0854afa913969c32dcc787c'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:12.950058 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-88672c14-ecd8-49a6-a3fd-b14fcf677029 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04580970d0854afa913969c32dcc787c'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:12.966307 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1133/2240] 10.222.0.98 () {58 vars in 967 bytes} [Thu Aug 18 12:22:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:12.978378 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-88672c14-ecd8-49a6-a3fd-b14fcf677029 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04580970d0854afa913969c32dcc787c'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:12.985906 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8552d8-3b55-4e43-ab91-8758e85bc9bf None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:12.991769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-2a8552d8-3b55-4e43-ab91-8758e85bc9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:13.073903 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1108/2241] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:22:12 2022] DELETE /identity/v3/users/04580970d0854afa913969c32dcc787c => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:22:13.148661 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8552d8-3b55-4e43-ab91-8758e85bc9bf None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:13.295265 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172277df-aaaf-467c-8df3-472688971aa5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771591fe31cb4e8c9c4320c2a7b2719d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:13.297133 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172277df-aaaf-467c-8df3-472688971aa5 service swift] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:13.297570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172277df-aaaf-467c-8df3-472688971aa5 service swift] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:13.298223 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172277df-aaaf-467c-8df3-472688971aa5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:13.395628 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172277df-aaaf-467c-8df3-472688971aa5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:13.398975 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172277df-aaaf-467c-8df3-472688971aa5 service swift] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:13.409693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1134/2242] 10.222.0.98 () {60 vars in 1384 bytes} [Thu Aug 18 12:22:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 12:22:13.418871 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:13.809895 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88672c14-ecd8-49a6-a3fd-b14fcf677029 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:13.811409 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-88672c14-ecd8-49a6-a3fd-b14fcf677029 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:13.891879 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88672c14-ecd8-49a6-a3fd-b14fcf677029 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:13.896808 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f4fd62-35c8-44cd-a19e-5079a7cf8d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:13.897444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f4fd62-35c8-44cd-a19e-5079a7cf8d34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:13.901596 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f4fd62-35c8-44cd-a19e-5079a7cf8d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:13.901596 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f4fd62-35c8-44cd-a19e-5079a7cf8d34 admin admin] PATH_INFO: `/v3/projects/e96813b43cff41eb995642099477b22d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:13.901596 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f4fd62-35c8-44cd-a19e-5079a7cf8d34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e96813b43cff41eb995642099477b22d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:13.901596 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f4fd62-35c8-44cd-a19e-5079a7cf8d34 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:13.971975 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:14.085847 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-d3f4fd62-35c8-44cd-a19e-5079a7cf8d34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e96813b43cff41eb995642099477b22d', 'request_id': 'req-d3f4fd62-35c8-44cd-a19e-5079a7cf8d34'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:14.097893 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-d3f4fd62-35c8-44cd-a19e-5079a7cf8d34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e96813b43cff41eb995642099477b22d', 'request_id': 'req-d3f4fd62-35c8-44cd-a19e-5079a7cf8d34'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:14.189590 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-d3f4fd62-35c8-44cd-a19e-5079a7cf8d34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e96813b43cff41eb995642099477b22d', 'request_id': 'req-d3f4fd62-35c8-44cd-a19e-5079a7cf8d34'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:14.207766 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-d3f4fd62-35c8-44cd-a19e-5079a7cf8d34 admin admin] The token cache is being invalidate because project e96813b43cff41eb995642099477b22d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:22:14.208091 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-d3f4fd62-35c8-44cd-a19e-5079a7cf8d34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:14.208919 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-d3f4fd62-35c8-44cd-a19e-5079a7cf8d34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:14.210401 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1109/2243] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:22:13 2022] DELETE /identity/v3/projects/e96813b43cff41eb995642099477b22d => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:22:14.215827 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d59141-e472-4bd1-9649-e61172621c84 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:14.216311 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d59141-e472-4bd1-9649-e61172621c84 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:14.216687 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d59141-e472-4bd1-9649-e61172621c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:14.249430 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-44d59141-e472-4bd1-9649-e61172621c84 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:14.250992 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-44d59141-e472-4bd1-9649-e61172621c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:14.355373 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1135/2244] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:14.365168 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2e84a-aa13-4c80-9e7e-192ba91206bd None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:14.367999 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2e84a-aa13-4c80-9e7e-192ba91206bd None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:14.369643 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2e84a-aa13-4c80-9e7e-192ba91206bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:14.421345 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-77a2e84a-aa13-4c80-9e7e-192ba91206bd None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:14.422604 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-77a2e84a-aa13-4c80-9e7e-192ba91206bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:14.434803 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1110/2245] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:14.441306 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d59141-e472-4bd1-9649-e61172621c84 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:14.456980 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d59141-e472-4bd1-9649-e61172621c84 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:14.469790 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2a39a-776a-4774-a017-613e174171c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:14.470454 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2a39a-776a-4774-a017-613e174171c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:14.470686 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2a39a-776a-4774-a017-613e174171c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:14.471492 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2a39a-776a-4774-a017-613e174171c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:14.471621 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc2a39a-776a-4774-a017-613e174171c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:22:14.471922 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc2a39a-776a-4774-a017-613e174171c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:14.480204 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc2a39a-776a-4774-a017-613e174171c7 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:14.519809 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1136/2246] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:22:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:22:14.524449 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a2e84a-aa13-4c80-9e7e-192ba91206bd None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:14.540873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a2e84a-aa13-4c80-9e7e-192ba91206bd None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:14.546375 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5cfa40-38c6-404f-82c6-76919b94b805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:14.549927 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5cfa40-38c6-404f-82c6-76919b94b805 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:14.549927 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5cfa40-38c6-404f-82c6-76919b94b805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:14.549927 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5cfa40-38c6-404f-82c6-76919b94b805 admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:14.549927 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5cfa40-38c6-404f-82c6-76919b94b805 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:14.550757 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5cfa40-38c6-404f-82c6-76919b94b805 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:14.715694 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1111/2247] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:22:14 2022] POST /identity/v3/projects => generated 375 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:22:14.716612 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2a39a-776a-4774-a017-613e174171c7 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:14.727713 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2a39a-776a-4774-a017-613e174171c7 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:14.809219 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942328e-d264-4f04-88d3-9dc175d7d66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:14.810061 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942328e-d264-4f04-88d3-9dc175d7d66a admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:14.810436 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942328e-d264-4f04-88d3-9dc175d7d66a admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:14.810806 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942328e-d264-4f04-88d3-9dc175d7d66a admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:14.811602 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e942328e-d264-4f04-88d3-9dc175d7d66a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:14.814048 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e942328e-d264-4f04-88d3-9dc175d7d66a admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:14.955123 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1137/2248] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:22:14 2022] POST /identity/v3/users => generated 288 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:22:14.956076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5cfa40-38c6-404f-82c6-76919b94b805 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:14.966969 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5cfa40-38c6-404f-82c6-76919b94b805 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:14.973672 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad41fbd-4215-44e5-b338-02d11dcb3555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:14.974288 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad41fbd-4215-44e5-b338-02d11dcb3555 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:14.974569 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad41fbd-4215-44e5-b338-02d11dcb3555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:14.974846 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad41fbd-4215-44e5-b338-02d11dcb3555 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:14.975536 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad41fbd-4215-44e5-b338-02d11dcb3555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:14.977149 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad41fbd-4215-44e5-b338-02d11dcb3555 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:15.009760 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942328e-d264-4f04-88d3-9dc175d7d66a admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:15.018794 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1112/2249] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 12:22:15.024154 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942328e-d264-4f04-88d3-9dc175d7d66a admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:15.029440 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e864e-52b0-4af4-b0e5-fce5d69deae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:15.030829 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e864e-52b0-4af4-b0e5-fce5d69deae5 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:15.031267 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e864e-52b0-4af4-b0e5-fce5d69deae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:15.031720 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e864e-52b0-4af4-b0e5-fce5d69deae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:15.032554 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9e864e-52b0-4af4-b0e5-fce5d69deae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:15.034502 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9e864e-52b0-4af4-b0e5-fce5d69deae5 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:15.053518 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad41fbd-4215-44e5-b338-02d11dcb3555 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:15.060770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1138/2250] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:22:15.064830 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad41fbd-4215-44e5-b338-02d11dcb3555 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:15.079718 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b18af1-5cbc-49ad-95db-cea10f53c24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:15.079718 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b18af1-5cbc-49ad-95db-cea10f53c24e admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:15.080333 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b18af1-5cbc-49ad-95db-cea10f53c24e admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:15.080829 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b18af1-5cbc-49ad-95db-cea10f53c24e admin admin] PATH_INFO: `/v3/projects/aab2b8df247049939462a8bb97eab5a9/users/cfb7a5edeb734c8290ff4cbff6858a7e/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:15.102613 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b18af1-5cbc-49ad-95db-cea10f53c24e admin admin] RBAC: Authorizing `identity:create_grant(project_id=aab2b8df247049939462a8bb97eab5a9, user_id=cfb7a5edeb734c8290ff4cbff6858a7e, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:15.106580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b18af1-5cbc-49ad-95db-cea10f53c24e admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:15.207764 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1113/2251] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:22:15 2022] PUT /identity/v3/projects/aab2b8df247049939462a8bb97eab5a9/users/cfb7a5edeb734c8290ff4cbff6858a7e/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:22:15.220448 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d607ce1-6c87-4b91-b2be-b2eb1743d342 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:15.220733 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d607ce1-6c87-4b91-b2be-b2eb1743d342 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:15.221219 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d607ce1-6c87-4b91-b2be-b2eb1743d342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:15.285502 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-5d607ce1-6c87-4b91-b2be-b2eb1743d342 None None] MFA Rules not processed for user `cfb7a5edeb734c8290ff4cbff6858a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:15.286981 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-5d607ce1-6c87-4b91-b2be-b2eb1743d342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:15.358001 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1139/2252] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:15.362698 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b18af1-5cbc-49ad-95db-cea10f53c24e admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:15.451991 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b18af1-5cbc-49ad-95db-cea10f53c24e admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:15.460002 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2bee9-896c-4092-a7b3-080daf5013f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:15.460748 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2bee9-896c-4092-a7b3-080daf5013f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:15.461104 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2bee9-896c-4092-a7b3-080daf5013f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:15.461460 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2bee9-896c-4092-a7b3-080daf5013f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:15.462183 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac2bee9-896c-4092-a7b3-080daf5013f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:15.464925 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac2bee9-896c-4092-a7b3-080daf5013f4 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:15.606446 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1114/2253] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:22:15 2022] POST /identity/v3/projects => generated 375 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:22:15.611931 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d607ce1-6c87-4b91-b2be-b2eb1743d342 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:15.626799 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d607ce1-6c87-4b91-b2be-b2eb1743d342 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:15.752024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5595e39-44e7-4def-bc54-00a41db9b96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:15.752991 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5595e39-44e7-4def-bc54-00a41db9b96d admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:15.755499 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5595e39-44e7-4def-bc54-00a41db9b96d admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:15.756690 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5595e39-44e7-4def-bc54-00a41db9b96d admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:15.758851 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:15.760837 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5595e39-44e7-4def-bc54-00a41db9b96d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:15.764323 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5595e39-44e7-4def-bc54-00a41db9b96d admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:15.903165 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1140/2254] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:22:15 2022] POST /identity/v3/users => generated 294 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:22:15.911364 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2bee9-896c-4092-a7b3-080daf5013f4 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:15.927269 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2bee9-896c-4092-a7b3-080daf5013f4 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:15.937528 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4431583-6118-4e9e-af18-be75ea77cace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:15.940438 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4431583-6118-4e9e-af18-be75ea77cace admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:15.940438 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4431583-6118-4e9e-af18-be75ea77cace admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:15.940438 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4431583-6118-4e9e-af18-be75ea77cace admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:15.940438 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4431583-6118-4e9e-af18-be75ea77cace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:15.944139 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4431583-6118-4e9e-af18-be75ea77cace admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:15.961912 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1115/2255] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:22:15.966737 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5595e39-44e7-4def-bc54-00a41db9b96d admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:15.979106 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5595e39-44e7-4def-bc54-00a41db9b96d admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:15.984908 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8ab45-1c32-4d13-94b4-dc3e9f8cff80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:15.988453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:15.988453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8ab45-1c32-4d13-94b4-dc3e9f8cff80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:15.988906 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8ab45-1c32-4d13-94b4-dc3e9f8cff80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:15.989327 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8ab45-1c32-4d13-94b4-dc3e9f8cff80 admin admin] PATH_INFO: `/v3/projects/f477de07b8884cc0bb4f862140b3f4e9/users/589f9d08e6cb437488c24d867b36a199/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:16.009907 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe8ab45-1c32-4d13-94b4-dc3e9f8cff80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f477de07b8884cc0bb4f862140b3f4e9, user_id=589f9d08e6cb437488c24d867b36a199, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:16.013821 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe8ab45-1c32-4d13-94b4-dc3e9f8cff80 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:16.107856 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1141/2256] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:22:15 2022] PUT /identity/v3/projects/f477de07b8884cc0bb4f862140b3f4e9/users/589f9d08e6cb437488c24d867b36a199/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:22:16.111866 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4431583-6118-4e9e-af18-be75ea77cace admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:16.129186 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa3f34-957b-4fdb-a011-5d6041b80e89 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:16.129474 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa3f34-957b-4fdb-a011-5d6041b80e89 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:16.129754 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa3f34-957b-4fdb-a011-5d6041b80e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:16.130469 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4431583-6118-4e9e-af18-be75ea77cace admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:16.157112 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-a6aa3f34-957b-4fdb-a011-5d6041b80e89 None None] MFA Rules not processed for user `d98949bd624a4330a473ebe5de31586a`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:16.158368 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-a6aa3f34-957b-4fdb-a011-5d6041b80e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:16.211877 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f15842-bf2c-4acb-a74d-833596ca8802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:16.213883 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f15842-bf2c-4acb-a74d-833596ca8802 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:16.213883 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f15842-bf2c-4acb-a74d-833596ca8802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:16.214253 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f15842-bf2c-4acb-a74d-833596ca8802 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:16.215145 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f15842-bf2c-4acb-a74d-833596ca8802 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:16.220008 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f15842-bf2c-4acb-a74d-833596ca8802 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:16.234873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1116/2257] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:22:16 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:22:16.239030 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f15842-bf2c-4acb-a74d-833596ca8802 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:16.241345 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1142/2258] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:22:16.252161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f15842-bf2c-4acb-a74d-833596ca8802 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:16.288079 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7995f-d4b4-407f-b284-150b3f350125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:16.296345 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7995f-d4b4-407f-b284-150b3f350125 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:16.297007 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7995f-d4b4-407f-b284-150b3f350125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:16.297394 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7995f-d4b4-407f-b284-150b3f350125 admin admin] PATH_INFO: `/v3/projects/f477de07b8884cc0bb4f862140b3f4e9/users/589f9d08e6cb437488c24d867b36a199/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:16.320388 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e7995f-d4b4-407f-b284-150b3f350125 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f477de07b8884cc0bb4f862140b3f4e9, user_id=589f9d08e6cb437488c24d867b36a199, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:16.336913 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e7995f-d4b4-407f-b284-150b3f350125 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:16.349143 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1117/2259] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:22:16 2022] PUT /identity/v3/projects/f477de07b8884cc0bb4f862140b3f4e9/users/589f9d08e6cb437488c24d867b36a199/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:22:16.361831 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a7ef18-3bfa-4e96-a66b-cba7732fd0fe None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:16.362316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a7ef18-3bfa-4e96-a66b-cba7732fd0fe None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:16.362770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a7ef18-3bfa-4e96-a66b-cba7732fd0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:16.411085 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-74a7ef18-3bfa-4e96-a66b-cba7732fd0fe None None] MFA Rules not processed for user `589f9d08e6cb437488c24d867b36a199`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:16.412348 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-74a7ef18-3bfa-4e96-a66b-cba7732fd0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:16.502574 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1143/2260] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:16 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:16.515105 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7995f-d4b4-407f-b284-150b3f350125 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:16.517568 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-46e7995f-d4b4-407f-b284-150b3f350125 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:16.589427 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7995f-d4b4-407f-b284-150b3f350125 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:16.593433 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e648d85-54b6-44ca-b466-1305e896f749 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:16.594052 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e648d85-54b6-44ca-b466-1305e896f749 service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:16.594333 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e648d85-54b6-44ca-b466-1305e896f749 service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:16.594602 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e648d85-54b6-44ca-b466-1305e896f749 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:16.664058 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e648d85-54b6-44ca-b466-1305e896f749 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:16.665922 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e648d85-54b6-44ca-b466-1305e896f749 service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:16.678459 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:16.684690 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1118/2261] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:22:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:22:17.369600 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a7ef18-3bfa-4e96-a66b-cba7732fd0fe None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:17.371054 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-74a7ef18-3bfa-4e96-a66b-cba7732fd0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:17.446045 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a7ef18-3bfa-4e96-a66b-cba7732fd0fe None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:17.452953 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af714a87-4793-4873-a6e9-30df744352a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:17.453613 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af714a87-4793-4873-a6e9-30df744352a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:17.453898 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af714a87-4793-4873-a6e9-30df744352a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:17.454179 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af714a87-4793-4873-a6e9-30df744352a7 admin admin] PATH_INFO: `/v3/users/991cdaf69fd5439c9cad12cc2be49ada` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:17.455318 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af714a87-4793-4873-a6e9-30df744352a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=991cdaf69fd5439c9cad12cc2be49ada)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:17.457321 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af714a87-4793-4873-a6e9-30df744352a7 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:17.701033 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-af714a87-4793-4873-a6e9-30df744352a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '991cdaf69fd5439c9cad12cc2be49ada'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:17.709117 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-af714a87-4793-4873-a6e9-30df744352a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '991cdaf69fd5439c9cad12cc2be49ada'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:17.709879 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-af714a87-4793-4873-a6e9-30df744352a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '991cdaf69fd5439c9cad12cc2be49ada'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:17.722849 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-af714a87-4793-4873-a6e9-30df744352a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '991cdaf69fd5439c9cad12cc2be49ada'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:17.801191 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-af714a87-4793-4873-a6e9-30df744352a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '991cdaf69fd5439c9cad12cc2be49ada'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:17.811290 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1144/2262] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:22:17 2022] DELETE /identity/v3/users/991cdaf69fd5439c9cad12cc2be49ada => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:22:17.815635 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1dabaa-dc39-4754-8ad3-8f64437dedbb None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:17.816193 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1dabaa-dc39-4754-8ad3-8f64437dedbb None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:17.816193 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1dabaa-dc39-4754-8ad3-8f64437dedbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:17.836824 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-2d1dabaa-dc39-4754-8ad3-8f64437dedbb None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:17.838143 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-2d1dabaa-dc39-4754-8ad3-8f64437dedbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:17.899063 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1119/2263] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:17.909802 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af714a87-4793-4873-a6e9-30df744352a7 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:17.911260 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-af714a87-4793-4873-a6e9-30df744352a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:17.946271 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:17.976195 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af714a87-4793-4873-a6e9-30df744352a7 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:17.978520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1dabaa-dc39-4754-8ad3-8f64437dedbb None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:17.980207 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-2d1dabaa-dc39-4754-8ad3-8f64437dedbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:17.980950 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2cb22e-0ae8-4449-9583-b17e1716a906 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:17.981585 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2cb22e-0ae8-4449-9583-b17e1716a906 service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:17.981866 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2cb22e-0ae8-4449-9583-b17e1716a906 service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:17.982139 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2cb22e-0ae8-4449-9583-b17e1716a906 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:17.991620 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2cb22e-0ae8-4449-9583-b17e1716a906 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:17.992378 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1dabaa-dc39-4754-8ad3-8f64437dedbb None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:17.993052 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2cb22e-0ae8-4449-9583-b17e1716a906 service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:17.997326 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50665f-5de1-4418-a2ff-44092570eff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:17.997955 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50665f-5de1-4418-a2ff-44092570eff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:17.998244 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50665f-5de1-4418-a2ff-44092570eff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:17.998521 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50665f-5de1-4418-a2ff-44092570eff3 admin admin] PATH_INFO: `/v3/users/6078e72ec68c4763ba8814812cc16fdd` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:17.999742 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad50665f-5de1-4418-a2ff-44092570eff3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6078e72ec68c4763ba8814812cc16fdd)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:18.001668 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad50665f-5de1-4418-a2ff-44092570eff3 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:18.003622 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1145/2264] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:22:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:22:18.261031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2cb22e-0ae8-4449-9583-b17e1716a906 service neutron] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:18.262439 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-6b2cb22e-0ae8-4449-9583-b17e1716a906 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:18.275860 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-ad50665f-5de1-4418-a2ff-44092570eff3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6078e72ec68c4763ba8814812cc16fdd'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:18.276625 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-ad50665f-5de1-4418-a2ff-44092570eff3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6078e72ec68c4763ba8814812cc16fdd'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:18.277336 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-ad50665f-5de1-4418-a2ff-44092570eff3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6078e72ec68c4763ba8814812cc16fdd'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:18.279959 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2cb22e-0ae8-4449-9583-b17e1716a906 service neutron] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:18.285761 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611917ef-aa5b-4107-bf53-83d2e19a4ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:18.286553 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611917ef-aa5b-4107-bf53-83d2e19a4ca2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:18.286912 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611917ef-aa5b-4107-bf53-83d2e19a4ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:18.287248 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611917ef-aa5b-4107-bf53-83d2e19a4ca2 admin admin] PATH_INFO: `/v3/projects/9b2edbbd46f54ad29ce12dad1762aaa7` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:18.288551 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611917ef-aa5b-4107-bf53-83d2e19a4ca2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b2edbbd46f54ad29ce12dad1762aaa7)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:18.290541 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611917ef-aa5b-4107-bf53-83d2e19a4ca2 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:18.301971 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-ad50665f-5de1-4418-a2ff-44092570eff3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6078e72ec68c4763ba8814812cc16fdd'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:18.318281 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-ad50665f-5de1-4418-a2ff-44092570eff3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6078e72ec68c4763ba8814812cc16fdd'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:18.430304 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1120/2265] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:22:17 2022] DELETE /identity/v3/users/6078e72ec68c4763ba8814812cc16fdd => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:22:18.435528 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aaee01-7499-4ec0-af88-8443a53a9882 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:18.435809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aaee01-7499-4ec0-af88-8443a53a9882 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:18.436845 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aaee01-7499-4ec0-af88-8443a53a9882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:18.461315 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-e6aaee01-7499-4ec0-af88-8443a53a9882 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:18.462241 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-e6aaee01-7499-4ec0-af88-8443a53a9882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:18.463685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-611917ef-aa5b-4107-bf53-83d2e19a4ca2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b2edbbd46f54ad29ce12dad1762aaa7', 'request_id': 'req-611917ef-aa5b-4107-bf53-83d2e19a4ca2'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:18.473223 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-611917ef-aa5b-4107-bf53-83d2e19a4ca2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b2edbbd46f54ad29ce12dad1762aaa7', 'request_id': 'req-611917ef-aa5b-4107-bf53-83d2e19a4ca2'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:18.538917 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1121/2266] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:18.549342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aaee01-7499-4ec0-af88-8443a53a9882 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:18.550780 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-e6aaee01-7499-4ec0-af88-8443a53a9882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:18.558307 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-611917ef-aa5b-4107-bf53-83d2e19a4ca2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b2edbbd46f54ad29ce12dad1762aaa7', 'request_id': 'req-611917ef-aa5b-4107-bf53-83d2e19a4ca2'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:18.572263 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-611917ef-aa5b-4107-bf53-83d2e19a4ca2 admin admin] The token cache is being invalidate because project 9b2edbbd46f54ad29ce12dad1762aaa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:22:18.577657 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-611917ef-aa5b-4107-bf53-83d2e19a4ca2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:18.577657 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-611917ef-aa5b-4107-bf53-83d2e19a4ca2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:18.577657 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1146/2267] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:22:18 2022] DELETE /identity/v3/projects/9b2edbbd46f54ad29ce12dad1762aaa7 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:22:18.581226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a31c53-b555-4ce0-bc5a-90f020d8997f None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:18.581908 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a31c53-b555-4ce0-bc5a-90f020d8997f None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:18.582504 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a31c53-b555-4ce0-bc5a-90f020d8997f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:18.604967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-a5a31c53-b555-4ce0-bc5a-90f020d8997f None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:18.606409 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-a5a31c53-b555-4ce0-bc5a-90f020d8997f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:18.615252 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1147/2268] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:22:18.623088 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b58f1-8186-4532-892c-9f4b45fa3b7e None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:18.624279 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b58f1-8186-4532-892c-9f4b45fa3b7e None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:18.624730 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b58f1-8186-4532-892c-9f4b45fa3b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:18.631007 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aaee01-7499-4ec0-af88-8443a53a9882 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:18.638661 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d7eb78-219d-4605-aadf-f8a303bbd3df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:18.640253 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7eb78-219d-4605-aadf-f8a303bbd3df service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:18.640373 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7eb78-219d-4605-aadf-f8a303bbd3df service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:18.640631 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7eb78-219d-4605-aadf-f8a303bbd3df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:18.643213 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-91d7eb78-219d-4605-aadf-f8a303bbd3df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:18.653208 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-099b58f1-8186-4532-892c-9f4b45fa3b7e None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:18.654525 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:18.655694 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-099b58f1-8186-4532-892c-9f4b45fa3b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:18.658956 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d7eb78-219d-4605-aadf-f8a303bbd3df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:18.660585 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d7eb78-219d-4605-aadf-f8a303bbd3df service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:18.667110 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1148/2269] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:22:18.672212 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b58f1-8186-4532-892c-9f4b45fa3b7e None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:18.691353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b58f1-8186-4532-892c-9f4b45fa3b7e None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:18.691543 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1122/2270] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:22:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:22:18.701267 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16cc433-e24a-4389-91b6-10876209db3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:18.702055 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cc433-e24a-4389-91b6-10876209db3a admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:18.705568 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cc433-e24a-4389-91b6-10876209db3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:18.705568 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cc433-e24a-4389-91b6-10876209db3a admin admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:18.705568 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16cc433-e24a-4389-91b6-10876209db3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:22:18.705568 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16cc433-e24a-4389-91b6-10876209db3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:18.707766 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16cc433-e24a-4389-91b6-10876209db3a admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:18.724682 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1149/2271] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:22:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:22:18.730300 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d7eb78-219d-4605-aadf-f8a303bbd3df service neutron] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:18.745422 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d7eb78-219d-4605-aadf-f8a303bbd3df service neutron] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:18.759053 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83063d31-495e-4553-8920-0f020dc55732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:18.759860 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83063d31-495e-4553-8920-0f020dc55732 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:18.760253 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83063d31-495e-4553-8920-0f020dc55732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:18.760648 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83063d31-495e-4553-8920-0f020dc55732 admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:18.761397 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83063d31-495e-4553-8920-0f020dc55732 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:18.763818 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83063d31-495e-4553-8920-0f020dc55732 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:18.916046 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1123/2272] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:22:18 2022] POST /identity/v3/projects => generated 383 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:22:18.916829 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16cc433-e24a-4389-91b6-10876209db3a admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:18.928641 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16cc433-e24a-4389-91b6-10876209db3a admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:19.012518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1811ac96-4386-47b5-80c8-633f28978b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:19.013153 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811ac96-4386-47b5-80c8-633f28978b27 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:19.013433 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811ac96-4386-47b5-80c8-633f28978b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:19.013715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811ac96-4386-47b5-80c8-633f28978b27 admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:19.014380 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1811ac96-4386-47b5-80c8-633f28978b27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:19.016419 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1811ac96-4386-47b5-80c8-633f28978b27 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:19.107741 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1150/2273] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:22:18 2022] POST /identity/v3/users => generated 292 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:22:19.112789 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83063d31-495e-4553-8920-0f020dc55732 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:19.123815 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83063d31-495e-4553-8920-0f020dc55732 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:19.129229 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f546403a-50a0-4fac-bee4-5edb46087afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:19.129850 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546403a-50a0-4fac-bee4-5edb46087afd admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:19.130133 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546403a-50a0-4fac-bee4-5edb46087afd admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:19.130411 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546403a-50a0-4fac-bee4-5edb46087afd admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:19.131083 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f546403a-50a0-4fac-bee4-5edb46087afd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:19.132727 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f546403a-50a0-4fac-bee4-5edb46087afd admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:19.143173 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1124/2274] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:22:19.147595 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1811ac96-4386-47b5-80c8-633f28978b27 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:19.167332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1811ac96-4386-47b5-80c8-633f28978b27 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:19.173034 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc87e0-9f72-4ce0-a6ba-c71d8e99de3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:19.174075 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc87e0-9f72-4ce0-a6ba-c71d8e99de3c admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:19.174553 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc87e0-9f72-4ce0-a6ba-c71d8e99de3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:19.174896 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc87e0-9f72-4ce0-a6ba-c71d8e99de3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:19.175881 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fc87e0-9f72-4ce0-a6ba-c71d8e99de3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:19.178728 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fc87e0-9f72-4ce0-a6ba-c71d8e99de3c admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:19.186698 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f546403a-50a0-4fac-bee4-5edb46087afd admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:19.190289 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-f546403a-50a0-4fac-bee4-5edb46087afd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:19.199812 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1151/2275] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:22:19.203898 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc87e0-9f72-4ce0-a6ba-c71d8e99de3c admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:19.215734 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc87e0-9f72-4ce0-a6ba-c71d8e99de3c admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:19.231501 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a1d7c4-d724-46cb-9581-7e105893c794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:19.231501 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1d7c4-d724-46cb-9581-7e105893c794 admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:19.231501 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1d7c4-d724-46cb-9581-7e105893c794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:19.231501 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1d7c4-d724-46cb-9581-7e105893c794 admin admin] PATH_INFO: `/v3/projects/c01ab18435804769bb4ae0bf86d4d99d/users/e6b767c363bb4a6fa3530f6a677e09f2/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:19.239805 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f546403a-50a0-4fac-bee4-5edb46087afd admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:19.249371 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318169e5-2914-45a5-823b-d3324d18df4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:19.250316 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318169e5-2914-45a5-823b-d3324d18df4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:19.250717 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318169e5-2914-45a5-823b-d3324d18df4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:19.250956 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318169e5-2914-45a5-823b-d3324d18df4d admin admin] PATH_INFO: `/v3/projects/2fb25d9e46be4c12824be2c46a5c264a` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:19.252780 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318169e5-2914-45a5-823b-d3324d18df4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fb25d9e46be4c12824be2c46a5c264a)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:19.254712 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318169e5-2914-45a5-823b-d3324d18df4d admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:19.263741 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a1d7c4-d724-46cb-9581-7e105893c794 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c01ab18435804769bb4ae0bf86d4d99d, user_id=e6b767c363bb4a6fa3530f6a677e09f2, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:19.268024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a1d7c4-d724-46cb-9581-7e105893c794 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:19.382191 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1152/2276] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:22:19 2022] PUT /identity/v3/projects/c01ab18435804769bb4ae0bf86d4d99d/users/e6b767c363bb4a6fa3530f6a677e09f2/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:22:19.384058 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb52cdb-3b5b-4389-a1ac-8637445d8825 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:19.384689 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb52cdb-3b5b-4389-a1ac-8637445d8825 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:19.384689 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb52cdb-3b5b-4389-a1ac-8637445d8825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:19.393866 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-318169e5-2914-45a5-823b-d3324d18df4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fb25d9e46be4c12824be2c46a5c264a', 'request_id': 'req-318169e5-2914-45a5-823b-d3324d18df4d'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:19.400056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-318169e5-2914-45a5-823b-d3324d18df4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fb25d9e46be4c12824be2c46a5c264a', 'request_id': 'req-318169e5-2914-45a5-823b-d3324d18df4d'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:19.418158 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-bcb52cdb-3b5b-4389-a1ac-8637445d8825 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:19.419829 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-bcb52cdb-3b5b-4389-a1ac-8637445d8825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:19.485896 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1153/2277] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:19.487198 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-318169e5-2914-45a5-823b-d3324d18df4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fb25d9e46be4c12824be2c46a5c264a', 'request_id': 'req-318169e5-2914-45a5-823b-d3324d18df4d'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:19.496034 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb52cdb-3b5b-4389-a1ac-8637445d8825 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:19.497478 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-bcb52cdb-3b5b-4389-a1ac-8637445d8825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:19.509640 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-318169e5-2914-45a5-823b-d3324d18df4d admin admin] The token cache is being invalidate because project 2fb25d9e46be4c12824be2c46a5c264a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:22:19.510162 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-318169e5-2914-45a5-823b-d3324d18df4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:19.511124 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-318169e5-2914-45a5-823b-d3324d18df4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:19.514933 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1125/2278] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:22:19 2022] DELETE /identity/v3/projects/2fb25d9e46be4c12824be2c46a5c264a => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:22:19.519748 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318169e5-2914-45a5-823b-d3324d18df4d admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:19.522148 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-318169e5-2914-45a5-823b-d3324d18df4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:19.541753 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318169e5-2914-45a5-823b-d3324d18df4d admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:19.548447 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8456ef39-8201-4323-8cf4-2889b8e26076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:19.549188 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8456ef39-8201-4323-8cf4-2889b8e26076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:19.549539 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8456ef39-8201-4323-8cf4-2889b8e26076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:19.549993 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8456ef39-8201-4323-8cf4-2889b8e26076 admin admin] PATH_INFO: `/v3/users/7bcd6f600e404142a98b8735d2671aa0` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:19.552012 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8456ef39-8201-4323-8cf4-2889b8e26076 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bcd6f600e404142a98b8735d2671aa0)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:19.554907 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8456ef39-8201-4323-8cf4-2889b8e26076 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:19.577695 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb52cdb-3b5b-4389-a1ac-8637445d8825 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:19.582919 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627adb63-b981-411a-9342-238b014f8c8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:19.582919 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627adb63-b981-411a-9342-238b014f8c8c service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:19.582919 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627adb63-b981-411a-9342-238b014f8c8c service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:19.582919 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627adb63-b981-411a-9342-238b014f8c8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:19.584878 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-627adb63-b981-411a-9342-238b014f8c8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:19.598667 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627adb63-b981-411a-9342-238b014f8c8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:19.600481 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627adb63-b981-411a-9342-238b014f8c8c service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:19.610778 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1154/2279] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:22:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:22:19.931949 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-8456ef39-8201-4323-8cf4-2889b8e26076 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bcd6f600e404142a98b8735d2671aa0'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:19.932611 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-8456ef39-8201-4323-8cf4-2889b8e26076 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bcd6f600e404142a98b8735d2671aa0'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:19.933211 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-8456ef39-8201-4323-8cf4-2889b8e26076 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bcd6f600e404142a98b8735d2671aa0'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:19.940835 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-8456ef39-8201-4323-8cf4-2889b8e26076 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bcd6f600e404142a98b8735d2671aa0'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:19.955669 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-8456ef39-8201-4323-8cf4-2889b8e26076 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bcd6f600e404142a98b8735d2671aa0'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:20.030824 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1126/2280] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:22:19 2022] DELETE /identity/v3/users/7bcd6f600e404142a98b8735d2671aa0 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:22:20.624581 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627adb63-b981-411a-9342-238b014f8c8c service neutron] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:20.645179 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627adb63-b981-411a-9342-238b014f8c8c service neutron] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:20.737661 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfac5c5-796c-466d-9978-41035705728e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:20.738518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfac5c5-796c-466d-9978-41035705728e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:20.738807 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfac5c5-796c-466d-9978-41035705728e admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:20.739090 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfac5c5-796c-466d-9978-41035705728e admin admin] PATH_INFO: `/v3/projects/40de2feaf64843c5bafd2c73dd6039a5` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:20.740542 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfac5c5-796c-466d-9978-41035705728e admin admin] RBAC: Authorizing `identity:delete_project(project_id=40de2feaf64843c5bafd2c73dd6039a5)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:20.742736 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfac5c5-796c-466d-9978-41035705728e admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:20.804473 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:20.893108 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-dbfac5c5-796c-466d-9978-41035705728e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40de2feaf64843c5bafd2c73dd6039a5', 'request_id': 'req-dbfac5c5-796c-466d-9978-41035705728e'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:20.899766 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-dbfac5c5-796c-466d-9978-41035705728e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40de2feaf64843c5bafd2c73dd6039a5', 'request_id': 'req-dbfac5c5-796c-466d-9978-41035705728e'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:20.969667 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-dbfac5c5-796c-466d-9978-41035705728e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40de2feaf64843c5bafd2c73dd6039a5', 'request_id': 'req-dbfac5c5-796c-466d-9978-41035705728e'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:20.983582 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-dbfac5c5-796c-466d-9978-41035705728e admin admin] The token cache is being invalidate because project 40de2feaf64843c5bafd2c73dd6039a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:22:20.983900 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-dbfac5c5-796c-466d-9978-41035705728e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:20.984617 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-dbfac5c5-796c-466d-9978-41035705728e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:20.986205 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1155/2281] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:22:20 2022] DELETE /identity/v3/projects/40de2feaf64843c5bafd2c73dd6039a5 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:22:21.047991 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:INSERT=1,keystone:SELECT=94,keystone:DELETE=5 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:21.213981 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae2688c-fe6c-46ba-a30f-ddefcf826b15 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:21.214266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae2688c-fe6c-46ba-a30f-ddefcf826b15 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:21.214547 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae2688c-fe6c-46ba-a30f-ddefcf826b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:21.245445 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-8ae2688c-fe6c-46ba-a30f-ddefcf826b15 None None] MFA Rules not processed for user `e6b767c363bb4a6fa3530f6a677e09f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:21.246686 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-8ae2688c-fe6c-46ba-a30f-ddefcf826b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:21.314401 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1127/2282] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:21.325051 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfac5c5-796c-466d-9978-41035705728e admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:21.326490 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-dbfac5c5-796c-466d-9978-41035705728e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:21.387947 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfac5c5-796c-466d-9978-41035705728e admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:21.395376 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4361b3a0-159f-4222-8869-1d4324c08dcb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:21.396018 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4361b3a0-159f-4222-8869-1d4324c08dcb service cinder] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:21.396299 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4361b3a0-159f-4222-8869-1d4324c08dcb service cinder] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:21.396628 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4361b3a0-159f-4222-8869-1d4324c08dcb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:21.407099 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4361b3a0-159f-4222-8869-1d4324c08dcb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:21.408575 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4361b3a0-159f-4222-8869-1d4324c08dcb service cinder] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:21.417453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1156/2283] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:22:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:22:21.900737 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:22.423677 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:22.792841 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:23.234987 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb73a5c-702d-4836-a315-a03b6ed260c0 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:23.235221 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb73a5c-702d-4836-a315-a03b6ed260c0 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:23.235531 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb73a5c-702d-4836-a315-a03b6ed260c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:23.258328 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-cdb73a5c-702d-4836-a315-a03b6ed260c0 None None] MFA Rules not processed for user `e6b767c363bb4a6fa3530f6a677e09f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:23.259575 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-cdb73a5c-702d-4836-a315-a03b6ed260c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:23.266407 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1128/2284] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:23.435879 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4361b3a0-159f-4222-8869-1d4324c08dcb service cinder] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:23.447042 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4361b3a0-159f-4222-8869-1d4324c08dcb service cinder] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:23.451456 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac9f17-dc20-4fc5-a5a3-5d5459d3b6a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:23.452582 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac9f17-dc20-4fc5-a5a3-5d5459d3b6a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:23.452862 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac9f17-dc20-4fc5-a5a3-5d5459d3b6a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:23.453139 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac9f17-dc20-4fc5-a5a3-5d5459d3b6a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:23.468864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-e7ac9f17-dc20-4fc5-a5a3-5d5459d3b6a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:23.547754 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ac9f17-dc20-4fc5-a5a3-5d5459d3b6a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:23.549234 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ac9f17-dc20-4fc5-a5a3-5d5459d3b6a2 service cinder] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:23.560633 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1157/2285] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:22:23 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:22:23.735710 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:23.878403 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:23.935337 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:24.453437 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:24.509622 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:25.119198 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:25.592754 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:26.000961 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:26.174807 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:26.358359 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:26.409522 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=80,keystone:INSERT=3,keystone:DELETE=17 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:30.318607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb73a5c-702d-4836-a315-a03b6ed260c0 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:30.320077 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-cdb73a5c-702d-4836-a315-a03b6ed260c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:30.400499 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb73a5c-702d-4836-a315-a03b6ed260c0 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:30.407170 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48853f73-b22f-40d4-92e4-d8e1329f313c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:30.408111 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48853f73-b22f-40d4-92e4-d8e1329f313c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:30.408732 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48853f73-b22f-40d4-92e4-d8e1329f313c admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:30.409036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48853f73-b22f-40d4-92e4-d8e1329f313c admin admin] PATH_INFO: `/v3/users/b0454173f6984ed69a2ea74c325378be` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:30.410958 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48853f73-b22f-40d4-92e4-d8e1329f313c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0454173f6984ed69a2ea74c325378be)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:30.413190 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48853f73-b22f-40d4-92e4-d8e1329f313c admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:30.644785 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-48853f73-b22f-40d4-92e4-d8e1329f313c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0454173f6984ed69a2ea74c325378be'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:30.645817 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-48853f73-b22f-40d4-92e4-d8e1329f313c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0454173f6984ed69a2ea74c325378be'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:30.646455 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-48853f73-b22f-40d4-92e4-d8e1329f313c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0454173f6984ed69a2ea74c325378be'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:30.656037 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-48853f73-b22f-40d4-92e4-d8e1329f313c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0454173f6984ed69a2ea74c325378be'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:30.672058 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-48853f73-b22f-40d4-92e4-d8e1329f313c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0454173f6984ed69a2ea74c325378be'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:30.875828 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1129/2286] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:22:30 2022] DELETE /identity/v3/users/b0454173f6984ed69a2ea74c325378be => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:22:30.875942 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db9a7e0-e936-4a54-8428-b056f223596f None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:30.875942 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db9a7e0-e936-4a54-8428-b056f223596f None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:30.875942 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db9a7e0-e936-4a54-8428-b056f223596f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:30.895583 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-6db9a7e0-e936-4a54-8428-b056f223596f None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:30.895583 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-6db9a7e0-e936-4a54-8428-b056f223596f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:30.955552 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1158/2287] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:30.966361 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48853f73-b22f-40d4-92e4-d8e1329f313c admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:30.978291 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-48853f73-b22f-40d4-92e4-d8e1329f313c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:31.032695 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:31.098138 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48853f73-b22f-40d4-92e4-d8e1329f313c admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:31.110365 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74759514-1a74-4508-8e09-e3e0f07eb0ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:31.112868 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74759514-1a74-4508-8e09-e3e0f07eb0ba service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:31.113578 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74759514-1a74-4508-8e09-e3e0f07eb0ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:31.114732 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74759514-1a74-4508-8e09-e3e0f07eb0ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:31.132663 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74759514-1a74-4508-8e09-e3e0f07eb0ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:31.138639 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74759514-1a74-4508-8e09-e3e0f07eb0ba service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:31.146062 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1130/2288] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:22:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:22:31.708370 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db9a7e0-e936-4a54-8428-b056f223596f None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:31.709443 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-6db9a7e0-e936-4a54-8428-b056f223596f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:31.722816 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db9a7e0-e936-4a54-8428-b056f223596f None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:31.727828 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a849d0fc-1d42-4274-bd7f-0675df79fbe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:31.728473 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849d0fc-1d42-4274-bd7f-0675df79fbe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:31.728863 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849d0fc-1d42-4274-bd7f-0675df79fbe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:31.729056 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849d0fc-1d42-4274-bd7f-0675df79fbe1 admin admin] PATH_INFO: `/v3/projects/76b91e9da2bd454a849168995269fee7` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:31.730193 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a849d0fc-1d42-4274-bd7f-0675df79fbe1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76b91e9da2bd454a849168995269fee7)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:31.732147 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a849d0fc-1d42-4274-bd7f-0675df79fbe1 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:31.866273 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:31.870125 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-a849d0fc-1d42-4274-bd7f-0675df79fbe1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76b91e9da2bd454a849168995269fee7', 'request_id': 'req-a849d0fc-1d42-4274-bd7f-0675df79fbe1'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:31.876227 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-a849d0fc-1d42-4274-bd7f-0675df79fbe1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76b91e9da2bd454a849168995269fee7', 'request_id': 'req-a849d0fc-1d42-4274-bd7f-0675df79fbe1'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:31.968765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-a849d0fc-1d42-4274-bd7f-0675df79fbe1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76b91e9da2bd454a849168995269fee7', 'request_id': 'req-a849d0fc-1d42-4274-bd7f-0675df79fbe1'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:31.981534 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-a849d0fc-1d42-4274-bd7f-0675df79fbe1 admin admin] The token cache is being invalidate because project 76b91e9da2bd454a849168995269fee7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:22:31.981969 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-a849d0fc-1d42-4274-bd7f-0675df79fbe1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:31.982743 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-a849d0fc-1d42-4274-bd7f-0675df79fbe1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:31.997940 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1159/2289] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:22:31 2022] DELETE /identity/v3/projects/76b91e9da2bd454a849168995269fee7 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:22:32.003203 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41eda0f-20db-47db-8521-c697fb566ead None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:32.009918 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41eda0f-20db-47db-8521-c697fb566ead None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:32.009918 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41eda0f-20db-47db-8521-c697fb566ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:32.031192 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-d41eda0f-20db-47db-8521-c697fb566ead None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:32.033489 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-d41eda0f-20db-47db-8521-c697fb566ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:32.101279 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1131/2290] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:32.107927 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3885a-ce69-431e-b574-57ccb4757111 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:32.108221 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3885a-ce69-431e-b574-57ccb4757111 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:32.108505 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3885a-ce69-431e-b574-57ccb4757111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:32.129910 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-fbb3885a-ce69-431e-b574-57ccb4757111 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:32.131264 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-fbb3885a-ce69-431e-b574-57ccb4757111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:32.140920 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1160/2291] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:32.145195 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41eda0f-20db-47db-8521-c697fb566ead None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:32.155006 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41eda0f-20db-47db-8521-c697fb566ead None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:32.159696 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e64d8c-6329-4aa6-bbae-20a69446ba65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:32.160250 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e64d8c-6329-4aa6-bbae-20a69446ba65 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:32.160535 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e64d8c-6329-4aa6-bbae-20a69446ba65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:32.160810 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e64d8c-6329-4aa6-bbae-20a69446ba65 admin admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:32.161483 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e64d8c-6329-4aa6-bbae-20a69446ba65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:22:32.161788 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e64d8c-6329-4aa6-bbae-20a69446ba65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:32.163382 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e64d8c-6329-4aa6-bbae-20a69446ba65 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:32.176093 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1132/2292] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:22:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:22:32.180455 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb3885a-ce69-431e-b574-57ccb4757111 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:32.190630 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb3885a-ce69-431e-b574-57ccb4757111 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:32.195662 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166138bf-bc9f-486e-a14b-e2008d736a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:32.196322 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166138bf-bc9f-486e-a14b-e2008d736a6c admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:32.196610 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166138bf-bc9f-486e-a14b-e2008d736a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:32.196884 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166138bf-bc9f-486e-a14b-e2008d736a6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:32.197512 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166138bf-bc9f-486e-a14b-e2008d736a6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:32.199402 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166138bf-bc9f-486e-a14b-e2008d736a6c admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:32.312589 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1161/2293] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:22:32 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:22:32.316822 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e64d8c-6329-4aa6-bbae-20a69446ba65 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:32.328599 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e64d8c-6329-4aa6-bbae-20a69446ba65 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:32.400992 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f8733-7944-4b4c-9d10-c692c62252da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:32.401620 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f8733-7944-4b4c-9d10-c692c62252da admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:32.401907 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f8733-7944-4b4c-9d10-c692c62252da admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:32.402196 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f8733-7944-4b4c-9d10-c692c62252da admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:32.402838 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f8733-7944-4b4c-9d10-c692c62252da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:32.404865 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f8733-7944-4b4c-9d10-c692c62252da admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:32.477803 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1133/2294] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:22:32 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:22:32.481785 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166138bf-bc9f-486e-a14b-e2008d736a6c admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:32.493222 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166138bf-bc9f-486e-a14b-e2008d736a6c admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:32.498703 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c9bd1f-319a-443a-bf70-cd3360df4bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:32.499875 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9bd1f-319a-443a-bf70-cd3360df4bdd admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:32.499875 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9bd1f-319a-443a-bf70-cd3360df4bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:32.500100 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9bd1f-319a-443a-bf70-cd3360df4bdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:32.500702 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c9bd1f-319a-443a-bf70-cd3360df4bdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:32.502310 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c9bd1f-319a-443a-bf70-cd3360df4bdd admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:32.512702 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1162/2295] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:22:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:22:32.516950 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f8733-7944-4b4c-9d10-c692c62252da admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:32.527308 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f8733-7944-4b4c-9d10-c692c62252da admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:32.532640 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b7b7c-ab6a-4808-801f-1e6f7a3b5191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:32.533298 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b7b7c-ab6a-4808-801f-1e6f7a3b5191 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:32.533605 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b7b7c-ab6a-4808-801f-1e6f7a3b5191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:32.534519 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b7b7c-ab6a-4808-801f-1e6f7a3b5191 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:32.534633 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797b7b7c-ab6a-4808-801f-1e6f7a3b5191 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:32.536171 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797b7b7c-ab6a-4808-801f-1e6f7a3b5191 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:32.547690 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1134/2296] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:22:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:22:32.551784 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c9bd1f-319a-443a-bf70-cd3360df4bdd admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:32.561942 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c9bd1f-319a-443a-bf70-cd3360df4bdd admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:32.566576 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b053ce5-4a39-4ebb-97ef-fb762cf23a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:32.567205 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b053ce5-4a39-4ebb-97ef-fb762cf23a14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:32.567524 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b053ce5-4a39-4ebb-97ef-fb762cf23a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:32.567799 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b053ce5-4a39-4ebb-97ef-fb762cf23a14 admin admin] PATH_INFO: `/v3/projects/944cd11144ca4cbf86962a3c9494ecbd/users/fcc7f1b599034654a3ed64dde2145bf8/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:32.582281 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b053ce5-4a39-4ebb-97ef-fb762cf23a14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=944cd11144ca4cbf86962a3c9494ecbd, user_id=fcc7f1b599034654a3ed64dde2145bf8, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:32.584996 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b053ce5-4a39-4ebb-97ef-fb762cf23a14 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:32.717889 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1163/2297] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:22:32 2022] PUT /identity/v3/projects/944cd11144ca4cbf86962a3c9494ecbd/users/fcc7f1b599034654a3ed64dde2145bf8/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:22:32.723021 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f9809-be35-47b4-9638-dc8a99355d58 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:32.723309 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f9809-be35-47b4-9638-dc8a99355d58 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:32.723634 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f9809-be35-47b4-9638-dc8a99355d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:32.746546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-182f9809-be35-47b4-9638-dc8a99355d58 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:32.746546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-182f9809-be35-47b4-9638-dc8a99355d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:32.768779 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:32.802856 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1135/2298] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:32.812966 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b053ce5-4a39-4ebb-97ef-fb762cf23a14 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:32.814747 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-9b053ce5-4a39-4ebb-97ef-fb762cf23a14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:32.871632 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b053ce5-4a39-4ebb-97ef-fb762cf23a14 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:32.875938 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7aa987-9964-4446-9528-a7a069c895ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:32.876546 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7aa987-9964-4446-9528-a7a069c895ce service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:32.877055 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7aa987-9964-4446-9528-a7a069c895ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:32.877148 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7aa987-9964-4446-9528-a7a069c895ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:32.886342 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7aa987-9964-4446-9528-a7a069c895ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:32.887807 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7aa987-9964-4446-9528-a7a069c895ce service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:32.897878 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1164/2299] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:22:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:22:40.121053 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005e295-6519-4668-8173-0663bf6eb8d8 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:40.122049 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005e295-6519-4668-8173-0663bf6eb8d8 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:40.122516 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005e295-6519-4668-8173-0663bf6eb8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:40.160729 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-0005e295-6519-4668-8173-0663bf6eb8d8 None None] MFA Rules not processed for user `2e8380a466484c83bca842518d171048`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:40.162867 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-0005e295-6519-4668-8173-0663bf6eb8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:40.267233 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1136/2300] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:40 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:40.392416 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b3267-8f21-4b11-8d06-d49076048df8 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:40.397526 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b3267-8f21-4b11-8d06-d49076048df8 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:40.397526 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b3267-8f21-4b11-8d06-d49076048df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:40.436277 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-6b4b3267-8f21-4b11-8d06-d49076048df8 None None] MFA Rules not processed for user `fcc7f1b599034654a3ed64dde2145bf8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:40.438359 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-6b4b3267-8f21-4b11-8d06-d49076048df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:40.536689 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1165/2301] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:40.541212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005e295-6519-4668-8173-0663bf6eb8d8 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:40.553259 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005e295-6519-4668-8173-0663bf6eb8d8 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:40.558760 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0930db-22ef-435a-aa5b-36e28f3770b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:40.563205 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0930db-22ef-435a-aa5b-36e28f3770b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:40.563205 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0930db-22ef-435a-aa5b-36e28f3770b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:40.563205 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0930db-22ef-435a-aa5b-36e28f3770b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:40.563205 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0930db-22ef-435a-aa5b-36e28f3770b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:40.563205 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0930db-22ef-435a-aa5b-36e28f3770b4 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:40.714846 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1137/2302] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:22:40 2022] POST /identity/v3/projects => generated 383 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:22:40.719099 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b3267-8f21-4b11-8d06-d49076048df8 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:40.725152 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:40.735125 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b3267-8f21-4b11-8d06-d49076048df8 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:40.807702 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b17ec3-c604-4472-92ee-dcd716b0e419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:40.808708 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b17ec3-c604-4472-92ee-dcd716b0e419 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:40.809095 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b17ec3-c604-4472-92ee-dcd716b0e419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:40.809476 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b17ec3-c604-4472-92ee-dcd716b0e419 admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:40.810521 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b17ec3-c604-4472-92ee-dcd716b0e419 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:40.813084 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b17ec3-c604-4472-92ee-dcd716b0e419 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:40.932784 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1166/2303] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:22:40 2022] POST /identity/v3/users => generated 298 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:22:40.933035 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0930db-22ef-435a-aa5b-36e28f3770b4 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:40.945248 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0930db-22ef-435a-aa5b-36e28f3770b4 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:40.951630 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3cadd1-fe9b-409e-bc75-5cbd337198dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:40.954510 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3cadd1-fe9b-409e-bc75-5cbd337198dd admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:40.954510 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3cadd1-fe9b-409e-bc75-5cbd337198dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:40.954510 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3cadd1-fe9b-409e-bc75-5cbd337198dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:40.955594 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3cadd1-fe9b-409e-bc75-5cbd337198dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:40.966863 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3cadd1-fe9b-409e-bc75-5cbd337198dd admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:40.983489 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1138/2304] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 12:22:40.985653 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b17ec3-c604-4472-92ee-dcd716b0e419 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:40.997064 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b17ec3-c604-4472-92ee-dcd716b0e419 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:41.003894 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca300057-7a7b-406b-affd-21493d2c7bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:41.008689 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca300057-7a7b-406b-affd-21493d2c7bea admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:41.008689 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca300057-7a7b-406b-affd-21493d2c7bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:41.008689 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca300057-7a7b-406b-affd-21493d2c7bea admin admin] PATH_INFO: `/v3/projects/1ee562f56a3b494e8c8edc09cb945508/users/d26989d779e645159c80b444394c2e66/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:41.024357 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca300057-7a7b-406b-affd-21493d2c7bea admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ee562f56a3b494e8c8edc09cb945508, user_id=d26989d779e645159c80b444394c2e66, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:41.028664 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca300057-7a7b-406b-affd-21493d2c7bea admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:41.157485 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1167/2305] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:22:40 2022] PUT /identity/v3/projects/1ee562f56a3b494e8c8edc09cb945508/users/d26989d779e645159c80b444394c2e66/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:22:41.161549 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3cadd1-fe9b-409e-bc75-5cbd337198dd admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:41.172372 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3cadd1-fe9b-409e-bc75-5cbd337198dd admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:41.280555 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635d6b4-bcf5-4e90-95de-8b522033f7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:41.281806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635d6b4-bcf5-4e90-95de-8b522033f7ff admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:41.281806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635d6b4-bcf5-4e90-95de-8b522033f7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:41.281806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635d6b4-bcf5-4e90-95de-8b522033f7ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:41.282481 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635d6b4-bcf5-4e90-95de-8b522033f7ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:41.284293 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635d6b4-bcf5-4e90-95de-8b522033f7ff admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:41.301003 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1139/2306] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:22:41 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 12:22:41.304537 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca300057-7a7b-406b-affd-21493d2c7bea admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:41.320260 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca300057-7a7b-406b-affd-21493d2c7bea admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:41.337316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500aafa-6780-48de-8fba-76d974b85c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:41.337316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500aafa-6780-48de-8fba-76d974b85c5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:41.337316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500aafa-6780-48de-8fba-76d974b85c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:41.337316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500aafa-6780-48de-8fba-76d974b85c5c admin admin] PATH_INFO: `/v3/projects/1ee562f56a3b494e8c8edc09cb945508/users/d26989d779e645159c80b444394c2e66/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:41.342676 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9500aafa-6780-48de-8fba-76d974b85c5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ee562f56a3b494e8c8edc09cb945508, user_id=d26989d779e645159c80b444394c2e66, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:41.342676 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9500aafa-6780-48de-8fba-76d974b85c5c admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:41.349681 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1168/2307] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:22:41 2022] PUT /identity/v3/projects/1ee562f56a3b494e8c8edc09cb945508/users/d26989d779e645159c80b444394c2e66/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:22:48.764278 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965f32a-13da-4c19-8b24-a420f5f617aa None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:48.765272 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965f32a-13da-4c19-8b24-a420f5f617aa None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:48.765805 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965f32a-13da-4c19-8b24-a420f5f617aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:48.803716 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-2965f32a-13da-4c19-8b24-a420f5f617aa None None] MFA Rules not processed for user `d26989d779e645159c80b444394c2e66`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:48.804988 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-2965f32a-13da-4c19-8b24-a420f5f617aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:48.844366 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:48.898151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1140/2308] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:48 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:48.912562 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500aafa-6780-48de-8fba-76d974b85c5c admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:48.915116 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-9500aafa-6780-48de-8fba-76d974b85c5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:48.983224 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500aafa-6780-48de-8fba-76d974b85c5c admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:48.987455 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a6418-6e88-47e4-b9e5-2cfb9d4cbfb2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd020063702b0481e9f4111680766f7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:48.988313 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a6418-6e88-47e4-b9e5-2cfb9d4cbfb2 service glance] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:48.988750 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a6418-6e88-47e4-b9e5-2cfb9d4cbfb2 service glance] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:48.989192 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a6418-6e88-47e4-b9e5-2cfb9d4cbfb2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:49.056440 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333a6418-6e88-47e4-b9e5-2cfb9d4cbfb2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:49.058098 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333a6418-6e88-47e4-b9e5-2cfb9d4cbfb2 service glance] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:49.067827 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1169/2309] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:22:48 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:22:49.368657 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8fc91-5ed1-44a4-8df4-49dad7456d05 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:49.368955 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8fc91-5ed1-44a4-8df4-49dad7456d05 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:49.369250 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8fc91-5ed1-44a4-8df4-49dad7456d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:49.390269 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-bad8fc91-5ed1-44a4-8df4-49dad7456d05 None None] MFA Rules not processed for user `b6880278f7304051b9ef03e927637e74`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:49.391523 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-bad8fc91-5ed1-44a4-8df4-49dad7456d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:49.453505 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1141/2310] 10.222.0.98 () {58 vars in 1009 bytes} [Thu Aug 18 12:22:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:49.463145 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a6418-6e88-47e4-b9e5-2cfb9d4cbfb2 service glance] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:49.464555 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-333a6418-6e88-47e4-b9e5-2cfb9d4cbfb2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:49.483571 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:49.544060 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a6418-6e88-47e4-b9e5-2cfb9d4cbfb2 service glance] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:49.549532 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fffc534-da7b-4735-9141-bad8f723d875 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771591fe31cb4e8c9c4320c2a7b2719d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:49.550152 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fffc534-da7b-4735-9141-bad8f723d875 service swift] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:49.550476 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fffc534-da7b-4735-9141-bad8f723d875 service swift] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:49.550740 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fffc534-da7b-4735-9141-bad8f723d875 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:49.559835 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fffc534-da7b-4735-9141-bad8f723d875 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:49.561504 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fffc534-da7b-4735-9141-bad8f723d875 service swift] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:49.574300 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1170/2311] 10.222.0.98 () {60 vars in 1384 bytes} [Thu Aug 18 12:22:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 12:22:55.470174 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad8fc91-5ed1-44a4-8df4-49dad7456d05 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:55.471663 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-bad8fc91-5ed1-44a4-8df4-49dad7456d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:55.596506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad8fc91-5ed1-44a4-8df4-49dad7456d05 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:55.596506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a051ee1f-3d95-44bc-9892-e20166624cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:55.596506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051ee1f-3d95-44bc-9892-e20166624cb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:55.596506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051ee1f-3d95-44bc-9892-e20166624cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:55.598153 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051ee1f-3d95-44bc-9892-e20166624cb9 admin admin] PATH_INFO: `/v3/users/42c92a39336b4127876efdef9b1e1763` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:55.598153 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a051ee1f-3d95-44bc-9892-e20166624cb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42c92a39336b4127876efdef9b1e1763)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:55.598153 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a051ee1f-3d95-44bc-9892-e20166624cb9 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:56.167048 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-a051ee1f-3d95-44bc-9892-e20166624cb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42c92a39336b4127876efdef9b1e1763'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:56.168231 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-a051ee1f-3d95-44bc-9892-e20166624cb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42c92a39336b4127876efdef9b1e1763'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:56.169987 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-a051ee1f-3d95-44bc-9892-e20166624cb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42c92a39336b4127876efdef9b1e1763'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:56.188280 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-a051ee1f-3d95-44bc-9892-e20166624cb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42c92a39336b4127876efdef9b1e1763'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:56.204201 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:56.210130 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-a051ee1f-3d95-44bc-9892-e20166624cb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42c92a39336b4127876efdef9b1e1763'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:56.294586 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1142/2312] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:22:55 2022] DELETE /identity/v3/users/42c92a39336b4127876efdef9b1e1763 => generated 0 bytes in 825 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:22:56.303165 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d3cb1-4116-4256-8ef7-fc9ea6e85cbb None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:56.303817 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d3cb1-4116-4256-8ef7-fc9ea6e85cbb None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:56.303817 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d3cb1-4116-4256-8ef7-fc9ea6e85cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:56.325353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-552d3cb1-4116-4256-8ef7-fc9ea6e85cbb None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:56.328173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-552d3cb1-4116-4256-8ef7-fc9ea6e85cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:56.409676 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1171/2313] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:56.420431 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a051ee1f-3d95-44bc-9892-e20166624cb9 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:56.422093 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-a051ee1f-3d95-44bc-9892-e20166624cb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:56.480922 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d3cb1-4116-4256-8ef7-fc9ea6e85cbb None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:56.483227 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-552d3cb1-4116-4256-8ef7-fc9ea6e85cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:56.508380 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d3cb1-4116-4256-8ef7-fc9ea6e85cbb None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:56.511709 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f0963-6eda-41a6-9c11-e8e98fadd1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:56.512535 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f0963-6eda-41a6-9c11-e8e98fadd1ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:56.513928 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f0963-6eda-41a6-9c11-e8e98fadd1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:56.513928 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f0963-6eda-41a6-9c11-e8e98fadd1ce admin admin] PATH_INFO: `/v3/users/cfb7a5edeb734c8290ff4cbff6858a7e` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:56.514764 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4f0963-6eda-41a6-9c11-e8e98fadd1ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfb7a5edeb734c8290ff4cbff6858a7e)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:56.519124 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4f0963-6eda-41a6-9c11-e8e98fadd1ce admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:56.543116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a051ee1f-3d95-44bc-9892-e20166624cb9 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:56.549073 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a20adc-eae6-4038-8fb9-367bc402875f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:56.550612 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a20adc-eae6-4038-8fb9-367bc402875f service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:56.551090 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a20adc-eae6-4038-8fb9-367bc402875f service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:56.551936 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a20adc-eae6-4038-8fb9-367bc402875f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:56.577216 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a20adc-eae6-4038-8fb9-367bc402875f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:56.577216 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a20adc-eae6-4038-8fb9-367bc402875f service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:56.581472 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1143/2314] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:22:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:22:56.792050 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-0a4f0963-6eda-41a6-9c11-e8e98fadd1ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfb7a5edeb734c8290ff4cbff6858a7e'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:56.801567 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-0a4f0963-6eda-41a6-9c11-e8e98fadd1ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfb7a5edeb734c8290ff4cbff6858a7e'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:56.806961 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-0a4f0963-6eda-41a6-9c11-e8e98fadd1ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfb7a5edeb734c8290ff4cbff6858a7e'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:56.819918 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-0a4f0963-6eda-41a6-9c11-e8e98fadd1ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfb7a5edeb734c8290ff4cbff6858a7e'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:56.896955 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-0a4f0963-6eda-41a6-9c11-e8e98fadd1ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfb7a5edeb734c8290ff4cbff6858a7e'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:56.904263 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1172/2315] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:22:56 2022] DELETE /identity/v3/users/cfb7a5edeb734c8290ff4cbff6858a7e => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:22:56.910709 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475533b2-c168-4b9f-84d4-0287bddd160f None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:56.911065 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475533b2-c168-4b9f-84d4-0287bddd160f None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:56.911549 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475533b2-c168-4b9f-84d4-0287bddd160f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:56.935143 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-475533b2-c168-4b9f-84d4-0287bddd160f None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:56.937041 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-475533b2-c168-4b9f-84d4-0287bddd160f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:57.033024 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1144/2316] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:57.040593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f0963-6eda-41a6-9c11-e8e98fadd1ce admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:57.042333 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-0a4f0963-6eda-41a6-9c11-e8e98fadd1ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:57.074341 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:57.230515 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f0963-6eda-41a6-9c11-e8e98fadd1ce admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:57.238065 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6049fc42-ca43-45ac-b4ae-cba782f9cea7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:57.238715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049fc42-ca43-45ac-b4ae-cba782f9cea7 service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:57.239102 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049fc42-ca43-45ac-b4ae-cba782f9cea7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:57.239388 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049fc42-ca43-45ac-b4ae-cba782f9cea7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:57.251695 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6049fc42-ca43-45ac-b4ae-cba782f9cea7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:57.258985 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6049fc42-ca43-45ac-b4ae-cba782f9cea7 service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:57.275678 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1173/2317] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:22:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:22:57.336545 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475533b2-c168-4b9f-84d4-0287bddd160f None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:57.338423 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-475533b2-c168-4b9f-84d4-0287bddd160f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:57.357002 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475533b2-c168-4b9f-84d4-0287bddd160f None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:57.379783 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9672fa-dcf9-4c88-a7aa-68774f505b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:57.380274 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9672fa-dcf9-4c88-a7aa-68774f505b1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:57.380726 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9672fa-dcf9-4c88-a7aa-68774f505b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:57.381002 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9672fa-dcf9-4c88-a7aa-68774f505b1c admin admin] PATH_INFO: `/v3/projects/53547de8e0b14177859608dda3893780` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:57.382169 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9672fa-dcf9-4c88-a7aa-68774f505b1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=53547de8e0b14177859608dda3893780)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:57.384190 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9672fa-dcf9-4c88-a7aa-68774f505b1c admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:57.581421 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-1e9672fa-dcf9-4c88-a7aa-68774f505b1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53547de8e0b14177859608dda3893780', 'request_id': 'req-1e9672fa-dcf9-4c88-a7aa-68774f505b1c'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:57.589584 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-1e9672fa-dcf9-4c88-a7aa-68774f505b1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53547de8e0b14177859608dda3893780', 'request_id': 'req-1e9672fa-dcf9-4c88-a7aa-68774f505b1c'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:57.774495 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-1e9672fa-dcf9-4c88-a7aa-68774f505b1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53547de8e0b14177859608dda3893780', 'request_id': 'req-1e9672fa-dcf9-4c88-a7aa-68774f505b1c'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:57.798924 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-1e9672fa-dcf9-4c88-a7aa-68774f505b1c admin admin] The token cache is being invalidate because project 53547de8e0b14177859608dda3893780 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:22:57.799332 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-1e9672fa-dcf9-4c88-a7aa-68774f505b1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:57.800050 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-1e9672fa-dcf9-4c88-a7aa-68774f505b1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:57.806845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f2296-421c-4658-8297-f1dd2503b78e None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:57.807278 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f2296-421c-4658-8297-f1dd2503b78e None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:57.807534 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f2296-421c-4658-8297-f1dd2503b78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:57.810754 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1145/2318] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:22:57 2022] DELETE /identity/v3/projects/53547de8e0b14177859608dda3893780 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:22:57.836933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-5f0f2296-421c-4658-8297-f1dd2503b78e None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:57.838176 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-5f0f2296-421c-4658-8297-f1dd2503b78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:57.937882 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1174/2319] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:57.940156 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9672fa-dcf9-4c88-a7aa-68774f505b1c admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:57.946260 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af41443-8ffe-47ce-b557-fff89dc659f6 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:57.947315 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af41443-8ffe-47ce-b557-fff89dc659f6 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:57.947636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af41443-8ffe-47ce-b557-fff89dc659f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:57.948977 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-1e9672fa-dcf9-4c88-a7aa-68774f505b1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:57.980934 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-0af41443-8ffe-47ce-b557-fff89dc659f6 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:57.982342 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-0af41443-8ffe-47ce-b557-fff89dc659f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:57.998937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1175/2320] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:58.000492 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af41443-8ffe-47ce-b557-fff89dc659f6 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:58.002158 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9672fa-dcf9-4c88-a7aa-68774f505b1c admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:58.007218 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a927d-f735-4f1b-a622-9a9575d18a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:58.008044 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a927d-f735-4f1b-a622-9a9575d18a03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:58.008963 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a927d-f735-4f1b-a622-9a9575d18a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:58.009235 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a927d-f735-4f1b-a622-9a9575d18a03 admin admin] PATH_INFO: `/v3/projects/aab2b8df247049939462a8bb97eab5a9` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:58.010649 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a927d-f735-4f1b-a622-9a9575d18a03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aab2b8df247049939462a8bb97eab5a9)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:58.013306 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a927d-f735-4f1b-a622-9a9575d18a03 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:58.014541 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af41443-8ffe-47ce-b557-fff89dc659f6 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:58.023810 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:58.025538 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ba6c44-0562-4d98-9f72-5801236b769e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:58.026184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ba6c44-0562-4d98-9f72-5801236b769e admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:58.026647 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ba6c44-0562-4d98-9f72-5801236b769e admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:58.026929 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ba6c44-0562-4d98-9f72-5801236b769e admin admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:58.027728 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ba6c44-0562-4d98-9f72-5801236b769e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:22:58.028629 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ba6c44-0562-4d98-9f72-5801236b769e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:58.030836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ba6c44-0562-4d98-9f72-5801236b769e admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:58.055788 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1176/2321] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:22:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:22:58.056800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ba6c44-0562-4d98-9f72-5801236b769e admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:58.070544 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ba6c44-0562-4d98-9f72-5801236b769e admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:58.076558 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e04d1b-50d2-4ec3-bc4b-c873c8e96073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:58.077310 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e04d1b-50d2-4ec3-bc4b-c873c8e96073 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:58.077703 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e04d1b-50d2-4ec3-bc4b-c873c8e96073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:58.078112 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e04d1b-50d2-4ec3-bc4b-c873c8e96073 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:58.079270 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e04d1b-50d2-4ec3-bc4b-c873c8e96073 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:58.083727 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e04d1b-50d2-4ec3-bc4b-c873c8e96073 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:58.154653 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-168a927d-f735-4f1b-a622-9a9575d18a03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aab2b8df247049939462a8bb97eab5a9', 'request_id': 'req-168a927d-f735-4f1b-a622-9a9575d18a03'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:58.165309 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-168a927d-f735-4f1b-a622-9a9575d18a03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aab2b8df247049939462a8bb97eab5a9', 'request_id': 'req-168a927d-f735-4f1b-a622-9a9575d18a03'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:58.223873 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1177/2322] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:22:58 2022] POST /identity/v3/projects => generated 397 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:22:58.227749 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e04d1b-50d2-4ec3-bc4b-c873c8e96073 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:58.245862 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-168a927d-f735-4f1b-a622-9a9575d18a03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aab2b8df247049939462a8bb97eab5a9', 'request_id': 'req-168a927d-f735-4f1b-a622-9a9575d18a03'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:58.271529 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-168a927d-f735-4f1b-a622-9a9575d18a03 admin admin] The token cache is being invalidate because project aab2b8df247049939462a8bb97eab5a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:22:58.271966 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-168a927d-f735-4f1b-a622-9a9575d18a03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:58.273785 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-168a927d-f735-4f1b-a622-9a9575d18a03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:58.277659 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1146/2323] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:22:57 2022] DELETE /identity/v3/projects/aab2b8df247049939462a8bb97eab5a9 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:22:58.281989 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a927d-f735-4f1b-a622-9a9575d18a03 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:58.284560 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-168a927d-f735-4f1b-a622-9a9575d18a03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:58.369144 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e04d1b-50d2-4ec3-bc4b-c873c8e96073 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:58.401602 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ca4a5-9a73-4f4d-ad30-ad77225bf89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:58.402583 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ca4a5-9a73-4f4d-ad30-ad77225bf89f admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:58.410047 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ca4a5-9a73-4f4d-ad30-ad77225bf89f admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:58.410559 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ca4a5-9a73-4f4d-ad30-ad77225bf89f admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:58.411371 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021ca4a5-9a73-4f4d-ad30-ad77225bf89f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:58.413663 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021ca4a5-9a73-4f4d-ad30-ad77225bf89f admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:58.431905 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a927d-f735-4f1b-a622-9a9575d18a03 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:58.442630 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc168ab3-4c77-4013-bdfe-589c2bbb3c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:58.443317 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc168ab3-4c77-4013-bdfe-589c2bbb3c2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:58.443652 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc168ab3-4c77-4013-bdfe-589c2bbb3c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:58.443935 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc168ab3-4c77-4013-bdfe-589c2bbb3c2c admin admin] PATH_INFO: `/v3/users/589f9d08e6cb437488c24d867b36a199` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:58.446799 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc168ab3-4c77-4013-bdfe-589c2bbb3c2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=589f9d08e6cb437488c24d867b36a199)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:58.450978 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc168ab3-4c77-4013-bdfe-589c2bbb3c2c admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:58.547721 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1178/2324] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:22:58 2022] POST /identity/v3/users => generated 299 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:22:58.555375 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ca4a5-9a73-4f4d-ad30-ad77225bf89f admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:58.557159 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-021ca4a5-9a73-4f4d-ad30-ad77225bf89f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:58.566921 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:58.595441 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ca4a5-9a73-4f4d-ad30-ad77225bf89f admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:58.602772 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4993728-406d-4975-8555-46eecb0ed0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:58.607587 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4993728-406d-4975-8555-46eecb0ed0e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:58.607587 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4993728-406d-4975-8555-46eecb0ed0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:58.607587 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4993728-406d-4975-8555-46eecb0ed0e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:58.608178 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4993728-406d-4975-8555-46eecb0ed0e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:58.610002 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4993728-406d-4975-8555-46eecb0ed0e6 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:58.626636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1179/2325] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:22:58.627843 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4993728-406d-4975-8555-46eecb0ed0e6 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:58.642499 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4993728-406d-4975-8555-46eecb0ed0e6 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:58.648565 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7ab60-24d6-4328-8285-44aadf114819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:58.649309 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7ab60-24d6-4328-8285-44aadf114819 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:58.649671 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7ab60-24d6-4328-8285-44aadf114819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:58.650584 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7ab60-24d6-4328-8285-44aadf114819 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:58.651331 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a7ab60-24d6-4328-8285-44aadf114819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:58.653197 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a7ab60-24d6-4328-8285-44aadf114819 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:58.668350 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1180/2326] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:22:58.682327 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7ab60-24d6-4328-8285-44aadf114819 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:58.711375 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7ab60-24d6-4328-8285-44aadf114819 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:58.723688 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a561f15-bc1c-4e52-922b-ecc7f2c423b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:58.724884 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a561f15-bc1c-4e52-922b-ecc7f2c423b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:58.725296 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a561f15-bc1c-4e52-922b-ecc7f2c423b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:58.725626 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-dc168ab3-4c77-4013-bdfe-589c2bbb3c2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '589f9d08e6cb437488c24d867b36a199'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:58.726078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a561f15-bc1c-4e52-922b-ecc7f2c423b3 admin admin] PATH_INFO: `/v3/projects/b8c0fe2cf4e748ec8887360781d66d0c/users/264843366bc34cf9ba83f5d52b1ba6dd/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:58.726783 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-dc168ab3-4c77-4013-bdfe-589c2bbb3c2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '589f9d08e6cb437488c24d867b36a199'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:58.727407 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-dc168ab3-4c77-4013-bdfe-589c2bbb3c2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '589f9d08e6cb437488c24d867b36a199'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:58.745238 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-dc168ab3-4c77-4013-bdfe-589c2bbb3c2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '589f9d08e6cb437488c24d867b36a199'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:58.764043 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-dc168ab3-4c77-4013-bdfe-589c2bbb3c2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '589f9d08e6cb437488c24d867b36a199'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:22:58.764173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a561f15-bc1c-4e52-922b-ecc7f2c423b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8c0fe2cf4e748ec8887360781d66d0c, user_id=264843366bc34cf9ba83f5d52b1ba6dd, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:58.766651 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a561f15-bc1c-4e52-922b-ecc7f2c423b3 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:58.864365 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1147/2327] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:22:58 2022] DELETE /identity/v3/users/589f9d08e6cb437488c24d867b36a199 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:22:58.867072 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1181/2328] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:22:58 2022] PUT /identity/v3/projects/b8c0fe2cf4e748ec8887360781d66d0c/users/264843366bc34cf9ba83f5d52b1ba6dd/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:22:58.878431 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0b64a-66c4-49ef-ab98-03e5c797f356 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:58.879119 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0b64a-66c4-49ef-ab98-03e5c797f356 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:58.879768 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0b64a-66c4-49ef-ab98-03e5c797f356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:58.927266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-77a0b64a-66c4-49ef-ab98-03e5c797f356 None None] MFA Rules not processed for user `264843366bc34cf9ba83f5d52b1ba6dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:22:58.929542 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-77a0b64a-66c4-49ef-ab98-03e5c797f356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:58.938048 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:22:59.056087 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1148/2329] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:22:58 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:22:59.060852 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a561f15-bc1c-4e52-922b-ecc7f2c423b3 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:59.060852 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-3a561f15-bc1c-4e52-922b-ecc7f2c423b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:59.142099 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a561f15-bc1c-4e52-922b-ecc7f2c423b3 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:59.222147 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d456bfe7-a504-440b-8b8d-6b77adce8c5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:59.222772 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456bfe7-a504-440b-8b8d-6b77adce8c5b service cinder] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:59.223050 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456bfe7-a504-440b-8b8d-6b77adce8c5b service cinder] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:59.223402 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456bfe7-a504-440b-8b8d-6b77adce8c5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:59.233916 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d456bfe7-a504-440b-8b8d-6b77adce8c5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:59.235535 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d456bfe7-a504-440b-8b8d-6b77adce8c5b service cinder] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:22:59.250129 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1182/2330] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:22:59 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:22:59.562949 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a0b64a-66c4-49ef-ab98-03e5c797f356 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:22:59.568568 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-77a0b64a-66c4-49ef-ab98-03e5c797f356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:22:59.728164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a0b64a-66c4-49ef-ab98-03e5c797f356 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:22:59.737131 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d88bf65-7f75-4173-8d72-56f3e800cec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:22:59.738614 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88bf65-7f75-4173-8d72-56f3e800cec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:22:59.739143 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88bf65-7f75-4173-8d72-56f3e800cec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:22:59.739703 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88bf65-7f75-4173-8d72-56f3e800cec4 admin admin] PATH_INFO: `/v3/projects/f477de07b8884cc0bb4f862140b3f4e9` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:22:59.745642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d88bf65-7f75-4173-8d72-56f3e800cec4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f477de07b8884cc0bb4f862140b3f4e9)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:22:59.748621 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d88bf65-7f75-4173-8d72-56f3e800cec4 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:00.394086 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-6d88bf65-7f75-4173-8d72-56f3e800cec4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f477de07b8884cc0bb4f862140b3f4e9', 'request_id': 'req-6d88bf65-7f75-4173-8d72-56f3e800cec4'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:00.401001 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-6d88bf65-7f75-4173-8d72-56f3e800cec4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f477de07b8884cc0bb4f862140b3f4e9', 'request_id': 'req-6d88bf65-7f75-4173-8d72-56f3e800cec4'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:00.591616 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-6d88bf65-7f75-4173-8d72-56f3e800cec4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f477de07b8884cc0bb4f862140b3f4e9', 'request_id': 'req-6d88bf65-7f75-4173-8d72-56f3e800cec4'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:00.609328 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-6d88bf65-7f75-4173-8d72-56f3e800cec4 admin admin] The token cache is being invalidate because project f477de07b8884cc0bb4f862140b3f4e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:23:00.609969 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-6d88bf65-7f75-4173-8d72-56f3e800cec4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:00.617259 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-6d88bf65-7f75-4173-8d72-56f3e800cec4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:00.627832 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1149/2331] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:22:59 2022] DELETE /identity/v3/projects/f477de07b8884cc0bb4f862140b3f4e9 => generated 0 bytes in 1066 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:23:00.632614 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9cd932-fa19-41da-933a-6df7065860d9 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:00.633142 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9cd932-fa19-41da-933a-6df7065860d9 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:00.633382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9cd932-fa19-41da-933a-6df7065860d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:00.674842 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-ac9cd932-fa19-41da-933a-6df7065860d9 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:23:00.686344 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-ac9cd932-fa19-41da-933a-6df7065860d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:00.765228 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1183/2332] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:23:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:23:00.768057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8a388-f238-4ae1-aa2f-466bcd4bf96f None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:00.768814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8a388-f238-4ae1-aa2f-466bcd4bf96f None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:00.768814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8a388-f238-4ae1-aa2f-466bcd4bf96f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:00.793696 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-0cc8a388-f238-4ae1-aa2f-466bcd4bf96f None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:23:00.800993 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-0cc8a388-f238-4ae1-aa2f-466bcd4bf96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:00.806191 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1150/2333] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:23:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:23:00.810206 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9cd932-fa19-41da-933a-6df7065860d9 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:00.822351 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9cd932-fa19-41da-933a-6df7065860d9 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:00.835552 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef063f6f-672c-471a-8595-53cd686760d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:00.836180 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef063f6f-672c-471a-8595-53cd686760d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:00.836462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef063f6f-672c-471a-8595-53cd686760d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:00.836735 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef063f6f-672c-471a-8595-53cd686760d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:00.837446 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef063f6f-672c-471a-8595-53cd686760d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:23:00.837684 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef063f6f-672c-471a-8595-53cd686760d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:00.844743 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef063f6f-672c-471a-8595-53cd686760d3 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:00.865068 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1184/2334] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:23:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:23:00.874838 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc8a388-f238-4ae1-aa2f-466bcd4bf96f None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:00.889751 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc8a388-f238-4ae1-aa2f-466bcd4bf96f None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:00.902068 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f99ab07-be40-464e-bfbc-1779e57e5f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:00.902702 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f99ab07-be40-464e-bfbc-1779e57e5f01 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:00.904242 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f99ab07-be40-464e-bfbc-1779e57e5f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:00.904527 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f99ab07-be40-464e-bfbc-1779e57e5f01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:00.905171 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f99ab07-be40-464e-bfbc-1779e57e5f01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:00.907742 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f99ab07-be40-464e-bfbc-1779e57e5f01 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:01.087040 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1151/2335] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:23:00 2022] POST /identity/v3/projects => generated 381 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:23:01.088793 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef063f6f-672c-471a-8595-53cd686760d3 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:01.094169 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:23:01.101917 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef063f6f-672c-471a-8595-53cd686760d3 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:01.176879 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642e00fb-f982-44af-9c29-4d184b71dd5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:01.177536 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e00fb-f982-44af-9c29-4d184b71dd5d admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:01.177932 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e00fb-f982-44af-9c29-4d184b71dd5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:01.178333 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e00fb-f982-44af-9c29-4d184b71dd5d admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:01.180137 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642e00fb-f982-44af-9c29-4d184b71dd5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:01.186970 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642e00fb-f982-44af-9c29-4d184b71dd5d admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:01.277559 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1185/2336] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:23:01 2022] POST /identity/v3/users => generated 291 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:23:01.281677 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f99ab07-be40-464e-bfbc-1779e57e5f01 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:01.296111 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f99ab07-be40-464e-bfbc-1779e57e5f01 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:01.301943 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e08da9-cfc3-4654-bdbf-263a664ad81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:01.307242 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e08da9-cfc3-4654-bdbf-263a664ad81f admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:01.307242 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e08da9-cfc3-4654-bdbf-263a664ad81f admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:01.307242 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e08da9-cfc3-4654-bdbf-263a664ad81f admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:01.307242 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e08da9-cfc3-4654-bdbf-263a664ad81f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:01.307242 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e08da9-cfc3-4654-bdbf-263a664ad81f admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:01.307242 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:23:01.319757 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1152/2337] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:23:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:23:01.323640 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642e00fb-f982-44af-9c29-4d184b71dd5d admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:01.335738 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642e00fb-f982-44af-9c29-4d184b71dd5d admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:01.344095 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa63d1-4eb1-4439-bb63-c69afa1bc0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:01.344788 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa63d1-4eb1-4439-bb63-c69afa1bc0e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:01.345171 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa63d1-4eb1-4439-bb63-c69afa1bc0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:01.345543 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa63d1-4eb1-4439-bb63-c69afa1bc0e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:01.346350 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa63d1-4eb1-4439-bb63-c69afa1bc0e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:01.348223 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa63d1-4eb1-4439-bb63-c69afa1bc0e7 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:01.365020 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1186/2338] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:23:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:23:01.369551 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e08da9-cfc3-4654-bdbf-263a664ad81f admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:01.385390 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e08da9-cfc3-4654-bdbf-263a664ad81f admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:01.391795 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8229b9-0a68-44fc-b148-731bbe34ae8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:01.392491 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8229b9-0a68-44fc-b148-731bbe34ae8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:01.392773 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8229b9-0a68-44fc-b148-731bbe34ae8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:01.393050 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8229b9-0a68-44fc-b148-731bbe34ae8b admin admin] PATH_INFO: `/v3/projects/5571ae053ca94ff9b19903414d402c8d/users/3eb33514e3284527a6108a6e86209864/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:01.419924 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8229b9-0a68-44fc-b148-731bbe34ae8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5571ae053ca94ff9b19903414d402c8d, user_id=3eb33514e3284527a6108a6e86209864, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:01.422970 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8229b9-0a68-44fc-b148-731bbe34ae8b admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:01.554069 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1153/2339] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:23:01 2022] PUT /identity/v3/projects/5571ae053ca94ff9b19903414d402c8d/users/3eb33514e3284527a6108a6e86209864/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:23:01.565470 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10598466-3c1e-41a3-b148-12a9dea4e232 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:01.566170 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10598466-3c1e-41a3-b148-12a9dea4e232 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:01.566658 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10598466-3c1e-41a3-b148-12a9dea4e232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:01.597530 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-10598466-3c1e-41a3-b148-12a9dea4e232 None None] MFA Rules not processed for user `3eb33514e3284527a6108a6e86209864`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:23:01.598916 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-10598466-3c1e-41a3-b148-12a9dea4e232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:01.682966 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1187/2340] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:23:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:23:01.688850 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8229b9-0a68-44fc-b148-731bbe34ae8b admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:01.771955 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8229b9-0a68-44fc-b148-731bbe34ae8b admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:01.776976 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b66c40-af41-41e3-b0f8-e4eb73a49b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:01.777602 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b66c40-af41-41e3-b0f8-e4eb73a49b65 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:01.778427 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b66c40-af41-41e3-b0f8-e4eb73a49b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:01.778711 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b66c40-af41-41e3-b0f8-e4eb73a49b65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:01.779344 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b66c40-af41-41e3-b0f8-e4eb73a49b65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:01.781401 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b66c40-af41-41e3-b0f8-e4eb73a49b65 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:01.911093 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1154/2341] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:23:01 2022] POST /identity/v3/projects => generated 383 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:23:01.912264 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10598466-3c1e-41a3-b148-12a9dea4e232 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:01.930573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10598466-3c1e-41a3-b148-12a9dea4e232 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:02.060330 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3a125-5b54-4226-b760-86da16f1e5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:02.061092 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3a125-5b54-4226-b760-86da16f1e5d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:02.061386 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3a125-5b54-4226-b760-86da16f1e5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:02.061663 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3a125-5b54-4226-b760-86da16f1e5d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:02.062301 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3a125-5b54-4226-b760-86da16f1e5d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:02.064280 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3a125-5b54-4226-b760-86da16f1e5d0 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:02.193559 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:23:02.202971 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1188/2342] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:23:01 2022] POST /identity/v3/users => generated 298 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:23:02.204530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b66c40-af41-41e3-b0f8-e4eb73a49b65 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:02.219012 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b66c40-af41-41e3-b0f8-e4eb73a49b65 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:02.227486 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75b7bee-bde1-4101-bc47-a45019ef6546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:02.227486 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b7bee-bde1-4101-bc47-a45019ef6546 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:02.227486 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b7bee-bde1-4101-bc47-a45019ef6546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:02.227486 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b7bee-bde1-4101-bc47-a45019ef6546 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:02.227927 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75b7bee-bde1-4101-bc47-a45019ef6546 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:02.229637 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75b7bee-bde1-4101-bc47-a45019ef6546 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:02.253705 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3a125-5b54-4226-b760-86da16f1e5d0 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:02.253832 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1155/2343] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:23:02 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:23:02.260688 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3a125-5b54-4226-b760-86da16f1e5d0 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:02.266144 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e5b2b-9f15-43a8-88bc-7dbe9c7dceef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:02.266835 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e5b2b-9f15-43a8-88bc-7dbe9c7dceef admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:02.267123 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e5b2b-9f15-43a8-88bc-7dbe9c7dceef admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:02.267442 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e5b2b-9f15-43a8-88bc-7dbe9c7dceef admin admin] PATH_INFO: `/v3/projects/18f26952b1df42f49b80576a92d40f30/users/f601e742745d4840a869d4fd00c63d31/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:02.287570 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7e5b2b-9f15-43a8-88bc-7dbe9c7dceef admin admin] RBAC: Authorizing `identity:create_grant(project_id=18f26952b1df42f49b80576a92d40f30, user_id=f601e742745d4840a869d4fd00c63d31, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:02.291075 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7e5b2b-9f15-43a8-88bc-7dbe9c7dceef admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:02.378775 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1189/2344] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:23:02 2022] PUT /identity/v3/projects/18f26952b1df42f49b80576a92d40f30/users/f601e742745d4840a869d4fd00c63d31/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:23:02.383471 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75b7bee-bde1-4101-bc47-a45019ef6546 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:02.396231 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75b7bee-bde1-4101-bc47-a45019ef6546 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:02.471190 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b981cbb-de27-42fd-9530-2adce906a046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:02.472325 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b981cbb-de27-42fd-9530-2adce906a046 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:02.472325 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b981cbb-de27-42fd-9530-2adce906a046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:02.477475 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b981cbb-de27-42fd-9530-2adce906a046 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:02.477475 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b981cbb-de27-42fd-9530-2adce906a046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:02.477475 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b981cbb-de27-42fd-9530-2adce906a046 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:02.487792 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1156/2345] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:23:02 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:23:02.492524 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e5b2b-9f15-43a8-88bc-7dbe9c7dceef admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:02.506354 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e5b2b-9f15-43a8-88bc-7dbe9c7dceef admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:02.511543 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f4992-88b9-4ad9-a798-c54bf8f813f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:02.512378 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f4992-88b9-4ad9-a798-c54bf8f813f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:02.512825 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f4992-88b9-4ad9-a798-c54bf8f813f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:02.513128 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f4992-88b9-4ad9-a798-c54bf8f813f7 admin admin] PATH_INFO: `/v3/projects/18f26952b1df42f49b80576a92d40f30/users/f601e742745d4840a869d4fd00c63d31/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:02.515126 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040f4992-88b9-4ad9-a798-c54bf8f813f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18f26952b1df42f49b80576a92d40f30, user_id=f601e742745d4840a869d4fd00c63d31, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:02.517864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040f4992-88b9-4ad9-a798-c54bf8f813f7 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:02.544107 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1190/2346] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:23:02 2022] PUT /identity/v3/projects/18f26952b1df42f49b80576a92d40f30/users/f601e742745d4840a869d4fd00c63d31/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:23:02.549880 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de11f1-cea0-46bf-9dd3-5a3dd29ec45f None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:02.550176 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de11f1-cea0-46bf-9dd3-5a3dd29ec45f None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:02.550773 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de11f1-cea0-46bf-9dd3-5a3dd29ec45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:02.586813 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-87de11f1-cea0-46bf-9dd3-5a3dd29ec45f None None] MFA Rules not processed for user `f601e742745d4840a869d4fd00c63d31`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:23:02.588108 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-87de11f1-cea0-46bf-9dd3-5a3dd29ec45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:02.627681 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:23:02.688695 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1157/2347] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:23:02 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:23:02.703794 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f4992-88b9-4ad9-a798-c54bf8f813f7 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:02.711407 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-040f4992-88b9-4ad9-a798-c54bf8f813f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:02.790675 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f4992-88b9-4ad9-a798-c54bf8f813f7 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:02.799066 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e1a2c-ac2e-4481-9736-04ce8a6f424f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:02.799811 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e1a2c-ac2e-4481-9736-04ce8a6f424f service cinder] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:02.800881 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e1a2c-ac2e-4481-9736-04ce8a6f424f service cinder] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:02.802751 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e1a2c-ac2e-4481-9736-04ce8a6f424f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:02.824836 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619e1a2c-ac2e-4481-9736-04ce8a6f424f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:02.826630 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619e1a2c-ac2e-4481-9736-04ce8a6f424f service cinder] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:02.841348 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1191/2348] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:23:02 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:23:02.961131 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de11f1-cea0-46bf-9dd3-5a3dd29ec45f None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:02.973159 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de11f1-cea0-46bf-9dd3-5a3dd29ec45f None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:02.982665 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b9aff-99e2-4ed8-a89a-40bed0741321 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:02.983327 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b9aff-99e2-4ed8-a89a-40bed0741321 service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:02.983650 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b9aff-99e2-4ed8-a89a-40bed0741321 service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:02.983971 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b9aff-99e2-4ed8-a89a-40bed0741321 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:03.063016 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b9aff-99e2-4ed8-a89a-40bed0741321 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:03.064908 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b9aff-99e2-4ed8-a89a-40bed0741321 service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:03.075921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1158/2349] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:23:02 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:23:05.356830 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a91ff-5244-4b1d-b526-d8c44a90646d None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:05.357431 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a91ff-5244-4b1d-b526-d8c44a90646d None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:05.357528 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a91ff-5244-4b1d-b526-d8c44a90646d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:05.377680 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-863a91ff-5244-4b1d-b526-d8c44a90646d None None] MFA Rules not processed for user `fcc7f1b599034654a3ed64dde2145bf8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:23:05.379060 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-863a91ff-5244-4b1d-b526-d8c44a90646d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:05.442742 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1192/2350] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:23:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:23:09.074126 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b9aff-99e2-4ed8-a89a-40bed0741321 service cinder] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:09.148621 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b9aff-99e2-4ed8-a89a-40bed0741321 service cinder] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:09.153785 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98d4dc-5b27-449b-9bbc-2c6a55ac6460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:09.154417 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98d4dc-5b27-449b-9bbc-2c6a55ac6460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:09.154706 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98d4dc-5b27-449b-9bbc-2c6a55ac6460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:09.154982 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98d4dc-5b27-449b-9bbc-2c6a55ac6460 admin admin] PATH_INFO: `/v3/users/3eb33514e3284527a6108a6e86209864` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:09.156147 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98d4dc-5b27-449b-9bbc-2c6a55ac6460 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3eb33514e3284527a6108a6e86209864)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:09.158067 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98d4dc-5b27-449b-9bbc-2c6a55ac6460 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:09.371555 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-ac98d4dc-5b27-449b-9bbc-2c6a55ac6460 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3eb33514e3284527a6108a6e86209864'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:09.372808 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-ac98d4dc-5b27-449b-9bbc-2c6a55ac6460 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3eb33514e3284527a6108a6e86209864'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:09.373485 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-ac98d4dc-5b27-449b-9bbc-2c6a55ac6460 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3eb33514e3284527a6108a6e86209864'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:09.385322 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-ac98d4dc-5b27-449b-9bbc-2c6a55ac6460 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3eb33514e3284527a6108a6e86209864'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:09.396354 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:23:09.406067 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-ac98d4dc-5b27-449b-9bbc-2c6a55ac6460 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3eb33514e3284527a6108a6e86209864'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:09.409301 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5d155-8348-4b5f-b31a-20f058e2a021 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:09.409814 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5d155-8348-4b5f-b31a-20f058e2a021 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:09.410136 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5d155-8348-4b5f-b31a-20f058e2a021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:09.433094 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-36b5d155-8348-4b5f-b31a-20f058e2a021 None None] MFA Rules not processed for user `b6880278f7304051b9ef03e927637e74`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:23:09.434917 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-36b5d155-8348-4b5f-b31a-20f058e2a021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:09.502871 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1193/2351] 10.222.0.98 () {58 vars in 1009 bytes} [Thu Aug 18 12:23:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:23:09.503796 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b5d155-8348-4b5f-b31a-20f058e2a021 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:09.505251 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-36b5d155-8348-4b5f-b31a-20f058e2a021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:09.527943 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1159/2352] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:23:09 2022] DELETE /identity/v3/users/3eb33514e3284527a6108a6e86209864 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:23:09.528931 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b5d155-8348-4b5f-b31a-20f058e2a021 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:09.531610 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98d4dc-5b27-449b-9bbc-2c6a55ac6460 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:09.533972 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-ac98d4dc-5b27-449b-9bbc-2c6a55ac6460 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:09.542002 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:23:09.592911 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38ee48-890b-4cb4-b178-4d5f3acf62d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:09.593553 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38ee48-890b-4cb4-b178-4d5f3acf62d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:09.593839 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38ee48-890b-4cb4-b178-4d5f3acf62d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:09.594126 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38ee48-890b-4cb4-b178-4d5f3acf62d8 admin admin] PATH_INFO: `/v3/users/264843366bc34cf9ba83f5d52b1ba6dd` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:09.595307 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38ee48-890b-4cb4-b178-4d5f3acf62d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=264843366bc34cf9ba83f5d52b1ba6dd)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:09.597284 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38ee48-890b-4cb4-b178-4d5f3acf62d8 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:09.609566 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98d4dc-5b27-449b-9bbc-2c6a55ac6460 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:09.615068 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf57e9-6634-4768-9b66-d3990883e3cf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771591fe31cb4e8c9c4320c2a7b2719d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:09.615715 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf57e9-6634-4768-9b66-d3990883e3cf service swift] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:09.615995 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf57e9-6634-4768-9b66-d3990883e3cf service swift] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:09.616538 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf57e9-6634-4768-9b66-d3990883e3cf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:09.702710 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedf57e9-6634-4768-9b66-d3990883e3cf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:09.710167 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedf57e9-6634-4768-9b66-d3990883e3cf service swift] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:09.724173 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1160/2353] 10.222.0.98 () {60 vars in 1384 bytes} [Thu Aug 18 12:23:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 12:23:09.725919 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048bbc3-d57b-4b49-bb60-3f551c2150d7 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:09.726122 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048bbc3-d57b-4b49-bb60-3f551c2150d7 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:09.726419 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048bbc3-d57b-4b49-bb60-3f551c2150d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:09.751634 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-e048bbc3-d57b-4b49-bb60-3f551c2150d7 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:23:09.752926 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-e048bbc3-d57b-4b49-bb60-3f551c2150d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:09.763255 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1161/2354] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:23:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:23:09.776449 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e048bbc3-d57b-4b49-bb60-3f551c2150d7 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:09.778244 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-e048bbc3-d57b-4b49-bb60-3f551c2150d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:09.900053 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e048bbc3-d57b-4b49-bb60-3f551c2150d7 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:09.906674 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ff0f9-acdf-44ea-b470-71840e0f13fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:09.907441 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ff0f9-acdf-44ea-b470-71840e0f13fb service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:09.908042 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ff0f9-acdf-44ea-b470-71840e0f13fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:09.909263 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ff0f9-acdf-44ea-b470-71840e0f13fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:09.910418 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-2f38ee48-890b-4cb4-b178-4d5f3acf62d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '264843366bc34cf9ba83f5d52b1ba6dd'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:09.932324 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-2f38ee48-890b-4cb4-b178-4d5f3acf62d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '264843366bc34cf9ba83f5d52b1ba6dd'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:09.935879 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-2f38ee48-890b-4cb4-b178-4d5f3acf62d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '264843366bc34cf9ba83f5d52b1ba6dd'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:09.961354 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-2f38ee48-890b-4cb4-b178-4d5f3acf62d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '264843366bc34cf9ba83f5d52b1ba6dd'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:09.962793 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ff0f9-acdf-44ea-b470-71840e0f13fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:09.964465 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ff0f9-acdf-44ea-b470-71840e0f13fb service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:09.978167 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1162/2355] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:23:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:23:10.044688 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-2f38ee48-890b-4cb4-b178-4d5f3acf62d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '264843366bc34cf9ba83f5d52b1ba6dd'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:10.057388 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1194/2356] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:23:09 2022] DELETE /identity/v3/users/264843366bc34cf9ba83f5d52b1ba6dd => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:23:10.062429 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfaf5b7-b1bf-4352-a245-1715612326a9 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:10.062783 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfaf5b7-b1bf-4352-a245-1715612326a9 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:10.063095 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfaf5b7-b1bf-4352-a245-1715612326a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:10.096069 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-9cfaf5b7-b1bf-4352-a245-1715612326a9 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:23:10.099633 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-9cfaf5b7-b1bf-4352-a245-1715612326a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:10.140142 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:23:10.243339 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1163/2357] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:23:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:23:10.257078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38ee48-890b-4cb4-b178-4d5f3acf62d8 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:10.258603 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-2f38ee48-890b-4cb4-b178-4d5f3acf62d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:10.356590 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38ee48-890b-4cb4-b178-4d5f3acf62d8 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:10.366898 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72fc5f6-101a-4516-a031-a182f0fd6414 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:10.370486 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72fc5f6-101a-4516-a031-a182f0fd6414 service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:10.370486 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72fc5f6-101a-4516-a031-a182f0fd6414 service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:10.370486 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72fc5f6-101a-4516-a031-a182f0fd6414 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:10.393012 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72fc5f6-101a-4516-a031-a182f0fd6414 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:10.394495 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72fc5f6-101a-4516-a031-a182f0fd6414 service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:10.407217 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1195/2358] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:23:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:23:10.904194 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfaf5b7-b1bf-4352-a245-1715612326a9 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:10.905637 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-9cfaf5b7-b1bf-4352-a245-1715612326a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:10.926007 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfaf5b7-b1bf-4352-a245-1715612326a9 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:10.942903 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd00cff-179c-47c5-a763-5ff91050cbed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:10.944626 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd00cff-179c-47c5-a763-5ff91050cbed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:10.946406 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd00cff-179c-47c5-a763-5ff91050cbed admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:10.946883 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd00cff-179c-47c5-a763-5ff91050cbed admin admin] PATH_INFO: `/v3/projects/5571ae053ca94ff9b19903414d402c8d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:10.949190 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd00cff-179c-47c5-a763-5ff91050cbed admin admin] RBAC: Authorizing `identity:delete_project(project_id=5571ae053ca94ff9b19903414d402c8d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:10.953143 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd00cff-179c-47c5-a763-5ff91050cbed admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:11.120380 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-6dd00cff-179c-47c5-a763-5ff91050cbed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5571ae053ca94ff9b19903414d402c8d', 'request_id': 'req-6dd00cff-179c-47c5-a763-5ff91050cbed'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:11.126483 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-6dd00cff-179c-47c5-a763-5ff91050cbed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5571ae053ca94ff9b19903414d402c8d', 'request_id': 'req-6dd00cff-179c-47c5-a763-5ff91050cbed'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:11.201717 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-6dd00cff-179c-47c5-a763-5ff91050cbed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5571ae053ca94ff9b19903414d402c8d', 'request_id': 'req-6dd00cff-179c-47c5-a763-5ff91050cbed'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:11.215988 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-6dd00cff-179c-47c5-a763-5ff91050cbed admin admin] The token cache is being invalidate because project 5571ae053ca94ff9b19903414d402c8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:23:11.216309 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-6dd00cff-179c-47c5-a763-5ff91050cbed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:11.216987 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-6dd00cff-179c-47c5-a763-5ff91050cbed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:11.218851 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1164/2359] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:23:10 2022] DELETE /identity/v3/projects/5571ae053ca94ff9b19903414d402c8d => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:23:11.222699 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72fc5f6-101a-4516-a031-a182f0fd6414 service neutron] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:11.233752 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72fc5f6-101a-4516-a031-a182f0fd6414 service neutron] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:11.270537 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:23:11.325720 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780bf3a-b279-4899-a2c3-54e6d6fdf211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:11.328503 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780bf3a-b279-4899-a2c3-54e6d6fdf211 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:11.328906 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780bf3a-b279-4899-a2c3-54e6d6fdf211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:11.329648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780bf3a-b279-4899-a2c3-54e6d6fdf211 admin admin] PATH_INFO: `/v3/users/f601e742745d4840a869d4fd00c63d31` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:11.331439 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5780bf3a-b279-4899-a2c3-54e6d6fdf211 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f601e742745d4840a869d4fd00c63d31)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:11.333845 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5780bf3a-b279-4899-a2c3-54e6d6fdf211 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:11.576088 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-5780bf3a-b279-4899-a2c3-54e6d6fdf211 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f601e742745d4840a869d4fd00c63d31'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:11.584837 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-5780bf3a-b279-4899-a2c3-54e6d6fdf211 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f601e742745d4840a869d4fd00c63d31'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:11.587835 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-5780bf3a-b279-4899-a2c3-54e6d6fdf211 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f601e742745d4840a869d4fd00c63d31'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:11.609088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd00cff-179c-47c5-a763-5ff91050cbed admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:11.612022 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-5780bf3a-b279-4899-a2c3-54e6d6fdf211 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f601e742745d4840a869d4fd00c63d31'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:11.612183 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-6dd00cff-179c-47c5-a763-5ff91050cbed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:11.629235 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd00cff-179c-47c5-a763-5ff91050cbed admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:11.641234 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81000956-b494-4dc5-bdcb-0a7ec9ad06e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:11.641997 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81000956-b494-4dc5-bdcb-0a7ec9ad06e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:11.642345 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81000956-b494-4dc5-bdcb-0a7ec9ad06e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:11.642689 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81000956-b494-4dc5-bdcb-0a7ec9ad06e6 admin admin] PATH_INFO: `/v3/projects/b8c0fe2cf4e748ec8887360781d66d0c` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:11.644079 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81000956-b494-4dc5-bdcb-0a7ec9ad06e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8c0fe2cf4e748ec8887360781d66d0c)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:11.646088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81000956-b494-4dc5-bdcb-0a7ec9ad06e6 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:11.704100 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-5780bf3a-b279-4899-a2c3-54e6d6fdf211 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f601e742745d4840a869d4fd00c63d31'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:11.710623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1196/2360] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:23:11 2022] DELETE /identity/v3/users/f601e742745d4840a869d4fd00c63d31 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:23:11.778719 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-81000956-b494-4dc5-bdcb-0a7ec9ad06e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8c0fe2cf4e748ec8887360781d66d0c', 'request_id': 'req-81000956-b494-4dc5-bdcb-0a7ec9ad06e6'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:11.786348 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-81000956-b494-4dc5-bdcb-0a7ec9ad06e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8c0fe2cf4e748ec8887360781d66d0c', 'request_id': 'req-81000956-b494-4dc5-bdcb-0a7ec9ad06e6'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:11.862175 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-81000956-b494-4dc5-bdcb-0a7ec9ad06e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8c0fe2cf4e748ec8887360781d66d0c', 'request_id': 'req-81000956-b494-4dc5-bdcb-0a7ec9ad06e6'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:11.875476 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-81000956-b494-4dc5-bdcb-0a7ec9ad06e6 admin admin] The token cache is being invalidate because project b8c0fe2cf4e748ec8887360781d66d0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:23:11.875740 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-81000956-b494-4dc5-bdcb-0a7ec9ad06e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:11.876372 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-81000956-b494-4dc5-bdcb-0a7ec9ad06e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:11.877885 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1165/2361] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:23:11 2022] DELETE /identity/v3/projects/b8c0fe2cf4e748ec8887360781d66d0c => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:23:12.225615 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780bf3a-b279-4899-a2c3-54e6d6fdf211 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:12.229396 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-5780bf3a-b279-4899-a2c3-54e6d6fdf211 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:12.307202 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780bf3a-b279-4899-a2c3-54e6d6fdf211 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:12.375210 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dedf9a-0f6d-4721-b28b-f8d92f32aab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:12.375974 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dedf9a-0f6d-4721-b28b-f8d92f32aab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:12.376329 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dedf9a-0f6d-4721-b28b-f8d92f32aab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:12.376672 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dedf9a-0f6d-4721-b28b-f8d92f32aab8 admin admin] PATH_INFO: `/v3/projects/18f26952b1df42f49b80576a92d40f30` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:12.378026 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dedf9a-0f6d-4721-b28b-f8d92f32aab8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18f26952b1df42f49b80576a92d40f30)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:12.380105 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dedf9a-0f6d-4721-b28b-f8d92f32aab8 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:12.387227 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:23:12.525159 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-87dedf9a-0f6d-4721-b28b-f8d92f32aab8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18f26952b1df42f49b80576a92d40f30', 'request_id': 'req-87dedf9a-0f6d-4721-b28b-f8d92f32aab8'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:12.526054 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-87dedf9a-0f6d-4721-b28b-f8d92f32aab8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18f26952b1df42f49b80576a92d40f30', 'request_id': 'req-87dedf9a-0f6d-4721-b28b-f8d92f32aab8'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:12.629665 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-87dedf9a-0f6d-4721-b28b-f8d92f32aab8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18f26952b1df42f49b80576a92d40f30', 'request_id': 'req-87dedf9a-0f6d-4721-b28b-f8d92f32aab8'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:12.643546 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-87dedf9a-0f6d-4721-b28b-f8d92f32aab8 admin admin] The token cache is being invalidate because project 18f26952b1df42f49b80576a92d40f30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:23:12.643854 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-87dedf9a-0f6d-4721-b28b-f8d92f32aab8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:12.644607 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-87dedf9a-0f6d-4721-b28b-f8d92f32aab8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:12.646733 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1197/2362] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:23:12 2022] DELETE /identity/v3/projects/18f26952b1df42f49b80576a92d40f30 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:23:12.651835 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe0ef13-3370-433d-8495-34b39f67bcde None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:12.656602 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe0ef13-3370-433d-8495-34b39f67bcde None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:12.656602 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe0ef13-3370-433d-8495-34b39f67bcde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:12.688422 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-dbe0ef13-3370-433d-8495-34b39f67bcde None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:23:12.691535 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-dbe0ef13-3370-433d-8495-34b39f67bcde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:12.722092 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:23:12.759223 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1166/2363] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:23:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:23:12.767799 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca4bfd-4382-4ac1-9acd-3f625ab5d3bd None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:12.771173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca4bfd-4382-4ac1-9acd-3f625ab5d3bd None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:12.771173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca4bfd-4382-4ac1-9acd-3f625ab5d3bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:12.788679 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-3bca4bfd-4382-4ac1-9acd-3f625ab5d3bd None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:23:12.789879 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-3bca4bfd-4382-4ac1-9acd-3f625ab5d3bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:12.798102 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1198/2364] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:23:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:23:12.803358 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe0ef13-3370-433d-8495-34b39f67bcde None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:12.816739 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe0ef13-3370-433d-8495-34b39f67bcde None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:12.822136 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88ea8e-7f31-4b17-a53e-fc7edab57169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:12.823491 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88ea8e-7f31-4b17-a53e-fc7edab57169 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:12.823897 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88ea8e-7f31-4b17-a53e-fc7edab57169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:12.824222 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88ea8e-7f31-4b17-a53e-fc7edab57169 admin admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:12.825418 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88ea8e-7f31-4b17-a53e-fc7edab57169 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:23:12.825848 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88ea8e-7f31-4b17-a53e-fc7edab57169 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:12.828392 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88ea8e-7f31-4b17-a53e-fc7edab57169 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:12.844174 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1167/2365] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:23:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:23:12.848620 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bca4bfd-4382-4ac1-9acd-3f625ab5d3bd None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:12.858905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bca4bfd-4382-4ac1-9acd-3f625ab5d3bd None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:12.863808 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1d701-8850-4e2f-a0a5-09961b3d2e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:12.864535 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1d701-8850-4e2f-a0a5-09961b3d2e4b admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:12.864879 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1d701-8850-4e2f-a0a5-09961b3d2e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:12.865471 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1d701-8850-4e2f-a0a5-09961b3d2e4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:12.866440 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1d701-8850-4e2f-a0a5-09961b3d2e4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:12.868633 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1d701-8850-4e2f-a0a5-09961b3d2e4b admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:12.981262 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1199/2366] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:23:12 2022] POST /identity/v3/projects => generated 371 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:23:12.985046 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88ea8e-7f31-4b17-a53e-fc7edab57169 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:12.995623 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88ea8e-7f31-4b17-a53e-fc7edab57169 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:13.052385 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6f063-ed8f-4fca-95b0-283fe2e009b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:13.052994 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6f063-ed8f-4fca-95b0-283fe2e009b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:13.053275 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6f063-ed8f-4fca-95b0-283fe2e009b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:13.053551 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6f063-ed8f-4fca-95b0-283fe2e009b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:13.054185 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6f063-ed8f-4fca-95b0-283fe2e009b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:13.056132 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6f063-ed8f-4fca-95b0-283fe2e009b7 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:13.137455 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1168/2367] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:23:12 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:23:13.142743 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1d701-8850-4e2f-a0a5-09961b3d2e4b admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:13.154498 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1d701-8850-4e2f-a0a5-09961b3d2e4b admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:13.159272 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d5189-3110-4084-a10e-911c9fbf4f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:13.160136 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d5189-3110-4084-a10e-911c9fbf4f76 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:13.160416 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d5189-3110-4084-a10e-911c9fbf4f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:13.160696 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d5189-3110-4084-a10e-911c9fbf4f76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:13.161337 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d5189-3110-4084-a10e-911c9fbf4f76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:13.162946 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d5189-3110-4084-a10e-911c9fbf4f76 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:13.173693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1200/2368] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:23:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:23:13.177819 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6f063-ed8f-4fca-95b0-283fe2e009b7 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:13.189682 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6f063-ed8f-4fca-95b0-283fe2e009b7 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:13.203850 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16055286-d7e9-4c0d-88c4-87ac51377dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:13.204769 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16055286-d7e9-4c0d-88c4-87ac51377dd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:13.205212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16055286-d7e9-4c0d-88c4-87ac51377dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:13.205664 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16055286-d7e9-4c0d-88c4-87ac51377dd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:13.206488 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16055286-d7e9-4c0d-88c4-87ac51377dd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:13.208414 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16055286-d7e9-4c0d-88c4-87ac51377dd1 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:13.221736 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1169/2369] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:23:13 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:23:13.225601 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d5189-3110-4084-a10e-911c9fbf4f76 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:13.236034 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d5189-3110-4084-a10e-911c9fbf4f76 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:13.240943 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4132dd80-1018-4a63-9f45-4addf3903f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:13.242027 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132dd80-1018-4a63-9f45-4addf3903f5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:13.242281 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132dd80-1018-4a63-9f45-4addf3903f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:13.242751 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132dd80-1018-4a63-9f45-4addf3903f5f admin admin] PATH_INFO: `/v3/projects/652129aa95a9417598b1cc4f5a45e16b/users/f27c39cbe15a4ee983a523d18a931eb6/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:13.259729 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4132dd80-1018-4a63-9f45-4addf3903f5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=652129aa95a9417598b1cc4f5a45e16b, user_id=f27c39cbe15a4ee983a523d18a931eb6, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:13.263076 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4132dd80-1018-4a63-9f45-4addf3903f5f admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:13.333611 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1201/2370] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:23:13 2022] PUT /identity/v3/projects/652129aa95a9417598b1cc4f5a45e16b/users/f27c39cbe15a4ee983a523d18a931eb6/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:23:13.344714 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356b5a8-22d3-4b81-99be-8069cc41e595 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:13.345081 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356b5a8-22d3-4b81-99be-8069cc41e595 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:13.346934 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356b5a8-22d3-4b81-99be-8069cc41e595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:13.378824 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-9356b5a8-22d3-4b81-99be-8069cc41e595 None None] MFA Rules not processed for user `f27c39cbe15a4ee983a523d18a931eb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:23:13.380097 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-9356b5a8-22d3-4b81-99be-8069cc41e595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:13.480355 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1170/2371] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:23:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:23:13.492226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4132dd80-1018-4a63-9f45-4addf3903f5f admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:13.493975 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-4132dd80-1018-4a63-9f45-4addf3903f5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:13.556430 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4132dd80-1018-4a63-9f45-4addf3903f5f admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:13.561254 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c4c05-ca9b-44f9-b667-8d23025f1fef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:13.561957 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c4c05-ca9b-44f9-b667-8d23025f1fef service cinder] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:13.562314 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c4c05-ca9b-44f9-b667-8d23025f1fef service cinder] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:13.563147 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c4c05-ca9b-44f9-b667-8d23025f1fef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:13.573096 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1c4c05-ca9b-44f9-b667-8d23025f1fef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:13.574638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1c4c05-ca9b-44f9-b667-8d23025f1fef service cinder] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:13.585052 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1202/2372] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:23:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:23:13.607904 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9356b5a8-22d3-4b81-99be-8069cc41e595 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:13.635383 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:23:13.678493 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9356b5a8-22d3-4b81-99be-8069cc41e595 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:13.695560 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3cd694-bf5b-42d3-a003-95de9a04951b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:13.695560 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3cd694-bf5b-42d3-a003-95de9a04951b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:13.695560 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3cd694-bf5b-42d3-a003-95de9a04951b admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:13.695560 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3cd694-bf5b-42d3-a003-95de9a04951b admin admin] PATH_INFO: `/v3/users/f27c39cbe15a4ee983a523d18a931eb6` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:13.695872 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3cd694-bf5b-42d3-a003-95de9a04951b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f27c39cbe15a4ee983a523d18a931eb6)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:13.695872 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3cd694-bf5b-42d3-a003-95de9a04951b admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:13.924806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-0d3cd694-bf5b-42d3-a003-95de9a04951b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f27c39cbe15a4ee983a523d18a931eb6'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:13.925481 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-0d3cd694-bf5b-42d3-a003-95de9a04951b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f27c39cbe15a4ee983a523d18a931eb6'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:13.926081 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-0d3cd694-bf5b-42d3-a003-95de9a04951b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f27c39cbe15a4ee983a523d18a931eb6'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:13.942693 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-0d3cd694-bf5b-42d3-a003-95de9a04951b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f27c39cbe15a4ee983a523d18a931eb6'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:13.955548 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-0d3cd694-bf5b-42d3-a003-95de9a04951b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f27c39cbe15a4ee983a523d18a931eb6'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:14.053819 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1171/2373] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:23:13 2022] DELETE /identity/v3/users/f27c39cbe15a4ee983a523d18a931eb6 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:23:14.058074 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534dd50e-5454-4ebb-8819-fdb2a45b66ad None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:14.058525 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534dd50e-5454-4ebb-8819-fdb2a45b66ad None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:14.058876 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534dd50e-5454-4ebb-8819-fdb2a45b66ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:14.079353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-534dd50e-5454-4ebb-8819-fdb2a45b66ad None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:23:14.080616 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-534dd50e-5454-4ebb-8819-fdb2a45b66ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:14.113106 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:23:14.149757 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1203/2374] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:23:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:23:14.156981 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3cd694-bf5b-42d3-a003-95de9a04951b admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:14.158498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-0d3cd694-bf5b-42d3-a003-95de9a04951b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:14.223964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3cd694-bf5b-42d3-a003-95de9a04951b admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:14.227520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ef9a0-14f4-4ba1-907b-3ce8153a89fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:14.228121 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ef9a0-14f4-4ba1-907b-3ce8153a89fb service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:14.228401 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ef9a0-14f4-4ba1-907b-3ce8153a89fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:14.228706 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ef9a0-14f4-4ba1-907b-3ce8153a89fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:14.237632 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091ef9a0-14f4-4ba1-907b-3ce8153a89fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:14.239057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091ef9a0-14f4-4ba1-907b-3ce8153a89fb service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:14.248664 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1172/2375] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:23:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:23:14.682091 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534dd50e-5454-4ebb-8819-fdb2a45b66ad None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:14.685065 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-534dd50e-5454-4ebb-8819-fdb2a45b66ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:14.699315 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534dd50e-5454-4ebb-8819-fdb2a45b66ad None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:14.705633 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c68c6-3cc9-423f-99e5-ff5753556af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:14.706681 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c68c6-3cc9-423f-99e5-ff5753556af2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:14.706681 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c68c6-3cc9-423f-99e5-ff5753556af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:14.706915 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c68c6-3cc9-423f-99e5-ff5753556af2 admin admin] PATH_INFO: `/v3/projects/652129aa95a9417598b1cc4f5a45e16b` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:14.709050 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984c68c6-3cc9-423f-99e5-ff5753556af2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=652129aa95a9417598b1cc4f5a45e16b)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:14.713046 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984c68c6-3cc9-423f-99e5-ff5753556af2 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:14.894503 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-984c68c6-3cc9-423f-99e5-ff5753556af2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '652129aa95a9417598b1cc4f5a45e16b', 'request_id': 'req-984c68c6-3cc9-423f-99e5-ff5753556af2'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:14.900761 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-984c68c6-3cc9-423f-99e5-ff5753556af2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '652129aa95a9417598b1cc4f5a45e16b', 'request_id': 'req-984c68c6-3cc9-423f-99e5-ff5753556af2'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:15.481338 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-984c68c6-3cc9-423f-99e5-ff5753556af2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '652129aa95a9417598b1cc4f5a45e16b', 'request_id': 'req-984c68c6-3cc9-423f-99e5-ff5753556af2'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:15.494265 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-984c68c6-3cc9-423f-99e5-ff5753556af2 admin admin] The token cache is being invalidate because project 652129aa95a9417598b1cc4f5a45e16b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:23:15.494695 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-984c68c6-3cc9-423f-99e5-ff5753556af2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:15.495341 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-984c68c6-3cc9-423f-99e5-ff5753556af2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:15.500945 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1204/2376] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:23:14 2022] DELETE /identity/v3/projects/652129aa95a9417598b1cc4f5a45e16b => generated 0 bytes in 816 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:23:15.503126 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac0005-54ce-4caa-b889-1ddec5bd6ead None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:15.504053 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac0005-54ce-4caa-b889-1ddec5bd6ead None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:15.504388 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac0005-54ce-4caa-b889-1ddec5bd6ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:15.525014 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-64ac0005-54ce-4caa-b889-1ddec5bd6ead None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:23:15.526236 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-64ac0005-54ce-4caa-b889-1ddec5bd6ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:15.571862 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:23:15.589803 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1173/2377] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:23:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:23:15.600520 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4397c6-4630-459d-a5a0-0a590f60b2a8 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:15.601064 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4397c6-4630-459d-a5a0-0a590f60b2a8 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:15.601600 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4397c6-4630-459d-a5a0-0a590f60b2a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:15.625094 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-eb4397c6-4630-459d-a5a0-0a590f60b2a8 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:23:15.626489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-eb4397c6-4630-459d-a5a0-0a590f60b2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:15.635098 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1205/2378] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:23:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:23:15.639476 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ac0005-54ce-4caa-b889-1ddec5bd6ead None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:15.649548 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ac0005-54ce-4caa-b889-1ddec5bd6ead None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:15.654389 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2bc001-0081-4eda-85b9-77931314d60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:15.655137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2bc001-0081-4eda-85b9-77931314d60d admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:15.655435 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2bc001-0081-4eda-85b9-77931314d60d admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:15.655746 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2bc001-0081-4eda-85b9-77931314d60d admin admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:15.656842 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2bc001-0081-4eda-85b9-77931314d60d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:23:15.657152 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2bc001-0081-4eda-85b9-77931314d60d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:15.658827 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2bc001-0081-4eda-85b9-77931314d60d admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:15.673618 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1174/2379] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:23:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:23:15.678096 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4397c6-4630-459d-a5a0-0a590f60b2a8 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:15.694334 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4397c6-4630-459d-a5a0-0a590f60b2a8 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:15.700677 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a26ae-9749-454c-870f-23fe898d50a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:15.703252 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a26ae-9749-454c-870f-23fe898d50a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:15.703610 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a26ae-9749-454c-870f-23fe898d50a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:15.704047 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a26ae-9749-454c-870f-23fe898d50a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:15.704553 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687a26ae-9749-454c-870f-23fe898d50a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:15.706451 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687a26ae-9749-454c-870f-23fe898d50a0 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:15.819138 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2bc001-0081-4eda-85b9-77931314d60d admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:15.831526 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1206/2380] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:23:15 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:23:15.834693 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2bc001-0081-4eda-85b9-77931314d60d admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:15.904109 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b764c-be3d-41b1-9836-0ff99943df1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:15.904734 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b764c-be3d-41b1-9836-0ff99943df1a admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:15.905027 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b764c-be3d-41b1-9836-0ff99943df1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:15.905312 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b764c-be3d-41b1-9836-0ff99943df1a admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:15.905949 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9b764c-be3d-41b1-9836-0ff99943df1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:15.908026 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9b764c-be3d-41b1-9836-0ff99943df1a admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:16.010552 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1175/2381] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:23:15 2022] POST /identity/v3/users => generated 292 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:23:16.015681 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a26ae-9749-454c-870f-23fe898d50a0 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:16.028326 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a26ae-9749-454c-870f-23fe898d50a0 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:16.036173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e859b58b-0f5b-4f57-b872-6bb9161e2bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:16.036173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859b58b-0f5b-4f57-b872-6bb9161e2bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:16.036173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859b58b-0f5b-4f57-b872-6bb9161e2bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:16.036173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859b58b-0f5b-4f57-b872-6bb9161e2bc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:16.036604 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e859b58b-0f5b-4f57-b872-6bb9161e2bc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:16.036774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e859b58b-0f5b-4f57-b872-6bb9161e2bc8 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:16.047641 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1207/2382] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:23:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:23:16.051917 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b764c-be3d-41b1-9836-0ff99943df1a admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:16.063228 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b764c-be3d-41b1-9836-0ff99943df1a admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:16.068416 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e54d0d-fe46-4973-bde7-181cfbf78a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:16.069034 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e54d0d-fe46-4973-bde7-181cfbf78a58 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:16.069380 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e54d0d-fe46-4973-bde7-181cfbf78a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:16.069599 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e54d0d-fe46-4973-bde7-181cfbf78a58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:16.070250 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e54d0d-fe46-4973-bde7-181cfbf78a58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:16.071909 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e54d0d-fe46-4973-bde7-181cfbf78a58 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:16.085476 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1176/2383] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:23:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:23:16.089869 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e859b58b-0f5b-4f57-b872-6bb9161e2bc8 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:16.105252 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e859b58b-0f5b-4f57-b872-6bb9161e2bc8 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:16.110864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10654946-2670-4b44-aa07-440af053a50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:16.111439 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10654946-2670-4b44-aa07-440af053a50f admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:16.111827 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10654946-2670-4b44-aa07-440af053a50f admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:16.112002 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10654946-2670-4b44-aa07-440af053a50f admin admin] PATH_INFO: `/v3/projects/6126a6dc378b4e6484be35033b23fee2/users/322d4a4233ef48379ed594120c6fafcb/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:16.128022 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10654946-2670-4b44-aa07-440af053a50f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6126a6dc378b4e6484be35033b23fee2, user_id=322d4a4233ef48379ed594120c6fafcb, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:16.130645 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10654946-2670-4b44-aa07-440af053a50f admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:16.228512 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1208/2384] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:23:16 2022] PUT /identity/v3/projects/6126a6dc378b4e6484be35033b23fee2/users/322d4a4233ef48379ed594120c6fafcb/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:23:16.230446 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ec7b6-4b4a-4407-8523-569ebf7e9f55 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:16.230734 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ec7b6-4b4a-4407-8523-569ebf7e9f55 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:16.231006 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ec7b6-4b4a-4407-8523-569ebf7e9f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:16.260294 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-ab4ec7b6-4b4a-4407-8523-569ebf7e9f55 None None] MFA Rules not processed for user `322d4a4233ef48379ed594120c6fafcb`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:23:16.261571 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-ab4ec7b6-4b4a-4407-8523-569ebf7e9f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:16.335227 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1177/2385] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:23:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:23:16.339720 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10654946-2670-4b44-aa07-440af053a50f admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:16.395685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:23:16.406758 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10654946-2670-4b44-aa07-440af053a50f admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:16.412337 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88ed91-665d-4adb-8ac2-e35365e4ddcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:16.412980 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88ed91-665d-4adb-8ac2-e35365e4ddcf admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:16.413267 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88ed91-665d-4adb-8ac2-e35365e4ddcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:16.413551 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88ed91-665d-4adb-8ac2-e35365e4ddcf admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:16.414211 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88ed91-665d-4adb-8ac2-e35365e4ddcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:16.416202 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88ed91-665d-4adb-8ac2-e35365e4ddcf admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:16.506433 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1209/2386] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:23:16 2022] POST /identity/v3/projects => generated 385 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:23:16.511062 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ec7b6-4b4a-4407-8523-569ebf7e9f55 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:16.522448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ec7b6-4b4a-4407-8523-569ebf7e9f55 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:16.569781 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:23:16.590996 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52369578-6ecf-4cd6-934e-dbf1b6d96586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:16.590996 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52369578-6ecf-4cd6-934e-dbf1b6d96586 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:16.590996 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52369578-6ecf-4cd6-934e-dbf1b6d96586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:16.590996 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52369578-6ecf-4cd6-934e-dbf1b6d96586 admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:16.591549 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52369578-6ecf-4cd6-934e-dbf1b6d96586 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:16.591549 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52369578-6ecf-4cd6-934e-dbf1b6d96586 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:16.720230 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1178/2387] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:23:16 2022] POST /identity/v3/users => generated 293 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:23:16.724239 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88ed91-665d-4adb-8ac2-e35365e4ddcf admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:16.735863 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88ed91-665d-4adb-8ac2-e35365e4ddcf admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:16.741704 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e6bc6-039d-49cf-9205-c8f84b900cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:16.742487 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e6bc6-039d-49cf-9205-c8f84b900cfe admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:16.742773 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e6bc6-039d-49cf-9205-c8f84b900cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:16.743739 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e6bc6-039d-49cf-9205-c8f84b900cfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:16.746053 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91e6bc6-039d-49cf-9205-c8f84b900cfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:16.746053 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91e6bc6-039d-49cf-9205-c8f84b900cfe admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:16.761637 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1210/2388] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:23:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:23:16.763525 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52369578-6ecf-4cd6-934e-dbf1b6d96586 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:16.774701 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52369578-6ecf-4cd6-934e-dbf1b6d96586 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:16.779856 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af01f69e-7a36-4e39-844d-9c3605b5a5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:16.780495 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01f69e-7a36-4e39-844d-9c3605b5a5f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:16.781140 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01f69e-7a36-4e39-844d-9c3605b5a5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:16.781432 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01f69e-7a36-4e39-844d-9c3605b5a5f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:16.782153 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af01f69e-7a36-4e39-844d-9c3605b5a5f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:16.783923 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af01f69e-7a36-4e39-844d-9c3605b5a5f5 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:16.796242 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1179/2389] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:23:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:23:16.800653 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e6bc6-039d-49cf-9205-c8f84b900cfe admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:16.811295 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e6bc6-039d-49cf-9205-c8f84b900cfe admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:16.817238 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0696540b-eb88-43e2-a4e7-faf7fd2fdcc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:16.817897 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696540b-eb88-43e2-a4e7-faf7fd2fdcc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:16.818184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696540b-eb88-43e2-a4e7-faf7fd2fdcc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:16.818476 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696540b-eb88-43e2-a4e7-faf7fd2fdcc1 admin admin] PATH_INFO: `/v3/projects/e2926cdf4e974648a080e429aac49e92/users/97763249c4f34211b54b571c840aee94/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:16.834250 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0696540b-eb88-43e2-a4e7-faf7fd2fdcc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2926cdf4e974648a080e429aac49e92, user_id=97763249c4f34211b54b571c840aee94, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:16.836933 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0696540b-eb88-43e2-a4e7-faf7fd2fdcc1 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:16.927313 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1211/2390] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:23:16 2022] PUT /identity/v3/projects/e2926cdf4e974648a080e429aac49e92/users/97763249c4f34211b54b571c840aee94/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:23:16.938831 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda45da-ea34-4b92-b2c5-ce926d4e07e3 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:16.939123 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda45da-ea34-4b92-b2c5-ce926d4e07e3 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:16.939401 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda45da-ea34-4b92-b2c5-ce926d4e07e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:16.983007 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-abda45da-ea34-4b92-b2c5-ce926d4e07e3 None None] MFA Rules not processed for user `97763249c4f34211b54b571c840aee94`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:23:16.984292 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-abda45da-ea34-4b92-b2c5-ce926d4e07e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:17.064024 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1180/2391] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:23:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:23:17.068226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0696540b-eb88-43e2-a4e7-faf7fd2fdcc1 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:17.132565 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0696540b-eb88-43e2-a4e7-faf7fd2fdcc1 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:17.143263 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38505456-d382-4552-a332-5de08c75c5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:17.143263 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38505456-d382-4552-a332-5de08c75c5cc admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:17.143263 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38505456-d382-4552-a332-5de08c75c5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:17.143263 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38505456-d382-4552-a332-5de08c75c5cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:17.143808 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38505456-d382-4552-a332-5de08c75c5cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:17.145758 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38505456-d382-4552-a332-5de08c75c5cc admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:17.264105 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1212/2392] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:23:17 2022] POST /identity/v3/projects => generated 383 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:23:17.268546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda45da-ea34-4b92-b2c5-ce926d4e07e3 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:17.280941 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda45da-ea34-4b92-b2c5-ce926d4e07e3 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:17.393911 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e177f96-27ce-4871-8a49-3d503ab4faee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:17.394546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e177f96-27ce-4871-8a49-3d503ab4faee admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:17.394901 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e177f96-27ce-4871-8a49-3d503ab4faee admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:17.395182 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e177f96-27ce-4871-8a49-3d503ab4faee admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:17.395911 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e177f96-27ce-4871-8a49-3d503ab4faee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:17.398257 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e177f96-27ce-4871-8a49-3d503ab4faee admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:17.494122 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1181/2393] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:23:17 2022] POST /identity/v3/users => generated 298 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:23:17.501092 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38505456-d382-4552-a332-5de08c75c5cc admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:17.511499 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38505456-d382-4552-a332-5de08c75c5cc admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:17.516212 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01043947-6b68-494d-a4da-ee17a0a346f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:17.516848 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01043947-6b68-494d-a4da-ee17a0a346f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:17.517084 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01043947-6b68-494d-a4da-ee17a0a346f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:17.517417 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01043947-6b68-494d-a4da-ee17a0a346f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:17.517974 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01043947-6b68-494d-a4da-ee17a0a346f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:17.519636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01043947-6b68-494d-a4da-ee17a0a346f1 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:17.532527 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1213/2394] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:23:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:23:17.538276 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e177f96-27ce-4871-8a49-3d503ab4faee admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:17.549160 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e177f96-27ce-4871-8a49-3d503ab4faee admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:17.553932 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7adf62-8a1b-4093-ab2f-0e870ba436d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:17.554609 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7adf62-8a1b-4093-ab2f-0e870ba436d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:17.554890 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7adf62-8a1b-4093-ab2f-0e870ba436d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:17.555166 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7adf62-8a1b-4093-ab2f-0e870ba436d7 admin admin] PATH_INFO: `/v3/projects/2cd7aebea8a34b2f847e8219762395ea/users/74fd797448f1479eb03bf66224f6d425/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:17.570477 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7adf62-8a1b-4093-ab2f-0e870ba436d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cd7aebea8a34b2f847e8219762395ea, user_id=74fd797448f1479eb03bf66224f6d425, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:17.575570 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7adf62-8a1b-4093-ab2f-0e870ba436d7 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:17.688543 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1182/2395] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:23:17 2022] PUT /identity/v3/projects/2cd7aebea8a34b2f847e8219762395ea/users/74fd797448f1479eb03bf66224f6d425/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:23:17.701513 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01043947-6b68-494d-a4da-ee17a0a346f1 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:17.718233 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01043947-6b68-494d-a4da-ee17a0a346f1 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:17.784179 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04fb30a-3472-40e2-aa68-bcbae8638db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:17.784908 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04fb30a-3472-40e2-aa68-bcbae8638db6 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:17.785070 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04fb30a-3472-40e2-aa68-bcbae8638db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:17.785345 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04fb30a-3472-40e2-aa68-bcbae8638db6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:17.785996 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04fb30a-3472-40e2-aa68-bcbae8638db6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:17.787620 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04fb30a-3472-40e2-aa68-bcbae8638db6 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:17.799192 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1214/2396] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:23:17 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:23:17.805340 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7adf62-8a1b-4093-ab2f-0e870ba436d7 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:17.817726 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7adf62-8a1b-4093-ab2f-0e870ba436d7 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:17.822837 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8062e15b-cdd8-479d-8cbb-b53f5fda055d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:17.823797 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062e15b-cdd8-479d-8cbb-b53f5fda055d admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:17.824226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062e15b-cdd8-479d-8cbb-b53f5fda055d admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:17.824514 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062e15b-cdd8-479d-8cbb-b53f5fda055d admin admin] PATH_INFO: `/v3/projects/2cd7aebea8a34b2f847e8219762395ea/users/74fd797448f1479eb03bf66224f6d425/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:17.826971 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8062e15b-cdd8-479d-8cbb-b53f5fda055d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cd7aebea8a34b2f847e8219762395ea, user_id=74fd797448f1479eb03bf66224f6d425, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:17.829744 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8062e15b-cdd8-479d-8cbb-b53f5fda055d admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:17.844161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1183/2397] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:23:17 2022] PUT /identity/v3/projects/2cd7aebea8a34b2f847e8219762395ea/users/74fd797448f1479eb03bf66224f6d425/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:23:17.915830 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926d1ec-45f5-4843-b195-b7b31de171fb None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:17.916079 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926d1ec-45f5-4843-b195-b7b31de171fb None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:17.916348 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926d1ec-45f5-4843-b195-b7b31de171fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:17.944395 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:23:17.952360 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-9926d1ec-45f5-4843-b195-b7b31de171fb None None] MFA Rules not processed for user `74fd797448f1479eb03bf66224f6d425`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:23:17.954302 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-9926d1ec-45f5-4843-b195-b7b31de171fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:18.047625 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1215/2398] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:23:17 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:23:18.068280 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8062e15b-cdd8-479d-8cbb-b53f5fda055d admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:18.068280 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-8062e15b-cdd8-479d-8cbb-b53f5fda055d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:18.071233 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:23:18.154211 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8062e15b-cdd8-479d-8cbb-b53f5fda055d admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:18.157358 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2659f-43aa-4c09-b28d-48984a146578 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:18.157970 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2659f-43aa-4c09-b28d-48984a146578 service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:18.158251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2659f-43aa-4c09-b28d-48984a146578 service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:18.158522 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2659f-43aa-4c09-b28d-48984a146578 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:18.241283 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da2659f-43aa-4c09-b28d-48984a146578 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:18.242775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da2659f-43aa-4c09-b28d-48984a146578 service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:18.254174 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1184/2399] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:23:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:23:20.251959 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9926d1ec-45f5-4843-b195-b7b31de171fb None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:20.253473 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-9926d1ec-45f5-4843-b195-b7b31de171fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:20.266713 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9926d1ec-45f5-4843-b195-b7b31de171fb None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:20.270459 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ace5a-ad97-44c0-8eaf-42c3db8fefac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:20.271151 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ace5a-ad97-44c0-8eaf-42c3db8fefac service cinder] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:20.271562 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ace5a-ad97-44c0-8eaf-42c3db8fefac service cinder] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:20.271925 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ace5a-ad97-44c0-8eaf-42c3db8fefac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:20.332667 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081ace5a-ad97-44c0-8eaf-42c3db8fefac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:20.334229 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081ace5a-ad97-44c0-8eaf-42c3db8fefac service cinder] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:20.344952 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1216/2400] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:23:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:23:21.689445 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2659f-43aa-4c09-b28d-48984a146578 service cinder] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:21.698167 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2659f-43aa-4c09-b28d-48984a146578 service cinder] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:21.701779 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e250ea4-ca15-40cf-8a1d-13ba36f420cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:21.702383 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e250ea4-ca15-40cf-8a1d-13ba36f420cd service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:21.702668 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e250ea4-ca15-40cf-8a1d-13ba36f420cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:21.702941 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e250ea4-ca15-40cf-8a1d-13ba36f420cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:21.712087 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e250ea4-ca15-40cf-8a1d-13ba36f420cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:21.714024 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e250ea4-ca15-40cf-8a1d-13ba36f420cd service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:21.723907 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1185/2401] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:23:21 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:23:26.372070 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ace5a-ad97-44c0-8eaf-42c3db8fefac service cinder] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:26.437325 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ace5a-ad97-44c0-8eaf-42c3db8fefac service cinder] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:26.442655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe7938-4e08-4dc9-a4d6-107ebfb978c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:26.443310 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe7938-4e08-4dc9-a4d6-107ebfb978c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:26.443684 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe7938-4e08-4dc9-a4d6-107ebfb978c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:26.443963 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe7938-4e08-4dc9-a4d6-107ebfb978c6 admin admin] PATH_INFO: `/v3/users/322d4a4233ef48379ed594120c6fafcb` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:26.445143 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe7938-4e08-4dc9-a4d6-107ebfb978c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=322d4a4233ef48379ed594120c6fafcb)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:26.447081 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe7938-4e08-4dc9-a4d6-107ebfb978c6 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:26.586523 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:23:26.592274 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-4cbe7938-4e08-4dc9-a4d6-107ebfb978c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '322d4a4233ef48379ed594120c6fafcb'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:26.599132 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-4cbe7938-4e08-4dc9-a4d6-107ebfb978c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '322d4a4233ef48379ed594120c6fafcb'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:26.603713 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-4cbe7938-4e08-4dc9-a4d6-107ebfb978c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '322d4a4233ef48379ed594120c6fafcb'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:26.612075 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-4cbe7938-4e08-4dc9-a4d6-107ebfb978c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '322d4a4233ef48379ed594120c6fafcb'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:26.733248 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-4cbe7938-4e08-4dc9-a4d6-107ebfb978c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '322d4a4233ef48379ed594120c6fafcb'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:26.743362 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1217/2402] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:23:26 2022] DELETE /identity/v3/users/322d4a4233ef48379ed594120c6fafcb => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:23:26.747856 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb01cd6-2e93-4f1a-9f1a-879bcd989f57 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:26.748874 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb01cd6-2e93-4f1a-9f1a-879bcd989f57 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:26.748968 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb01cd6-2e93-4f1a-9f1a-879bcd989f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:26.769988 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-1eb01cd6-2e93-4f1a-9f1a-879bcd989f57 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:23:26.771223 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-1eb01cd6-2e93-4f1a-9f1a-879bcd989f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:26.836881 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1186/2403] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:23:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:23:26.848558 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe7938-4e08-4dc9-a4d6-107ebfb978c6 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:26.850111 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-4cbe7938-4e08-4dc9-a4d6-107ebfb978c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:26.927267 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe7938-4e08-4dc9-a4d6-107ebfb978c6 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:26.931200 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ddf42e-3c5f-4e82-9589-887fbfefbd31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:26.931862 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ddf42e-3c5f-4e82-9589-887fbfefbd31 service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:26.932148 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ddf42e-3c5f-4e82-9589-887fbfefbd31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:26.932424 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ddf42e-3c5f-4e82-9589-887fbfefbd31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:26.943197 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ddf42e-3c5f-4e82-9589-887fbfefbd31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:26.945373 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ddf42e-3c5f-4e82-9589-887fbfefbd31 service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:26.955896 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1218/2404] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:23:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:23:27.333624 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb01cd6-2e93-4f1a-9f1a-879bcd989f57 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:27.335070 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-1eb01cd6-2e93-4f1a-9f1a-879bcd989f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:27.352720 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb01cd6-2e93-4f1a-9f1a-879bcd989f57 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:27.359273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad4534-a6bd-4f86-8904-0e912371a794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:27.359966 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad4534-a6bd-4f86-8904-0e912371a794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:27.360250 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad4534-a6bd-4f86-8904-0e912371a794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:27.360526 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad4534-a6bd-4f86-8904-0e912371a794 admin admin] PATH_INFO: `/v3/projects/6126a6dc378b4e6484be35033b23fee2` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:27.361677 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ad4534-a6bd-4f86-8904-0e912371a794 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6126a6dc378b4e6484be35033b23fee2)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:27.363617 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ad4534-a6bd-4f86-8904-0e912371a794 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:27.457751 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:23:27.464706 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-60ad4534-a6bd-4f86-8904-0e912371a794 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6126a6dc378b4e6484be35033b23fee2', 'request_id': 'req-60ad4534-a6bd-4f86-8904-0e912371a794'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:27.470035 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-60ad4534-a6bd-4f86-8904-0e912371a794 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6126a6dc378b4e6484be35033b23fee2', 'request_id': 'req-60ad4534-a6bd-4f86-8904-0e912371a794'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:27.520364 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-60ad4534-a6bd-4f86-8904-0e912371a794 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6126a6dc378b4e6484be35033b23fee2', 'request_id': 'req-60ad4534-a6bd-4f86-8904-0e912371a794'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:27.532347 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-60ad4534-a6bd-4f86-8904-0e912371a794 admin admin] The token cache is being invalidate because project 6126a6dc378b4e6484be35033b23fee2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:23:27.532840 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-60ad4534-a6bd-4f86-8904-0e912371a794 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:27.534151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-60ad4534-a6bd-4f86-8904-0e912371a794 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:27.535672 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1187/2405] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:23:27 2022] DELETE /identity/v3/projects/6126a6dc378b4e6484be35033b23fee2 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:23:27.539757 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ddf42e-3c5f-4e82-9589-887fbfefbd31 service neutron] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:27.550258 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ddf42e-3c5f-4e82-9589-887fbfefbd31 service neutron] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:27.607094 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c225182-f090-48cb-a316-63661702a511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:27.608188 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c225182-f090-48cb-a316-63661702a511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:27.608476 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c225182-f090-48cb-a316-63661702a511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:27.608755 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c225182-f090-48cb-a316-63661702a511 admin admin] PATH_INFO: `/v3/users/97763249c4f34211b54b571c840aee94` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:27.609893 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c225182-f090-48cb-a316-63661702a511 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97763249c4f34211b54b571c840aee94)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:27.611929 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c225182-f090-48cb-a316-63661702a511 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:27.815314 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-5c225182-f090-48cb-a316-63661702a511 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97763249c4f34211b54b571c840aee94'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:27.822243 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-5c225182-f090-48cb-a316-63661702a511 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97763249c4f34211b54b571c840aee94'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:27.822915 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-5c225182-f090-48cb-a316-63661702a511 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97763249c4f34211b54b571c840aee94'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:27.827498 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:23:27.835794 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-5c225182-f090-48cb-a316-63661702a511 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97763249c4f34211b54b571c840aee94'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:27.892214 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-5c225182-f090-48cb-a316-63661702a511 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97763249c4f34211b54b571c840aee94'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:27.899165 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1219/2406] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:23:27 2022] DELETE /identity/v3/users/97763249c4f34211b54b571c840aee94 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:23:28.400850 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad4534-a6bd-4f86-8904-0e912371a794 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:28.402480 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-60ad4534-a6bd-4f86-8904-0e912371a794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:28.416394 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad4534-a6bd-4f86-8904-0e912371a794 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:28.478002 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a38ac-1394-440d-93ca-c37adccdfa67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:28.478727 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a38ac-1394-440d-93ca-c37adccdfa67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:28.479017 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a38ac-1394-440d-93ca-c37adccdfa67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:28.479302 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a38ac-1394-440d-93ca-c37adccdfa67 admin admin] PATH_INFO: `/v3/projects/e2926cdf4e974648a080e429aac49e92` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:28.480636 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1a38ac-1394-440d-93ca-c37adccdfa67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2926cdf4e974648a080e429aac49e92)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:28.482632 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1a38ac-1394-440d-93ca-c37adccdfa67 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:28.623390 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-0b1a38ac-1394-440d-93ca-c37adccdfa67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2926cdf4e974648a080e429aac49e92', 'request_id': 'req-0b1a38ac-1394-440d-93ca-c37adccdfa67'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:28.629555 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-0b1a38ac-1394-440d-93ca-c37adccdfa67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2926cdf4e974648a080e429aac49e92', 'request_id': 'req-0b1a38ac-1394-440d-93ca-c37adccdfa67'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:28.707305 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-0b1a38ac-1394-440d-93ca-c37adccdfa67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2926cdf4e974648a080e429aac49e92', 'request_id': 'req-0b1a38ac-1394-440d-93ca-c37adccdfa67'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:28.719659 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-0b1a38ac-1394-440d-93ca-c37adccdfa67 admin admin] The token cache is being invalidate because project e2926cdf4e974648a080e429aac49e92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:23:28.719951 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-0b1a38ac-1394-440d-93ca-c37adccdfa67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:28.720612 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-0b1a38ac-1394-440d-93ca-c37adccdfa67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:28.722109 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1188/2407] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:23:28 2022] DELETE /identity/v3/projects/e2926cdf4e974648a080e429aac49e92 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:23:28.726152 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c225182-f090-48cb-a316-63661702a511 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:28.727832 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-5c225182-f090-48cb-a316-63661702a511 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:28.741999 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c225182-f090-48cb-a316-63661702a511 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:28.815664 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1fa54-276a-4c67-825f-7067807bdb57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:28.816330 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1fa54-276a-4c67-825f-7067807bdb57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:28.816618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1fa54-276a-4c67-825f-7067807bdb57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:28.817249 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1fa54-276a-4c67-825f-7067807bdb57 admin admin] PATH_INFO: `/v3/users/74fd797448f1479eb03bf66224f6d425` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:28.818530 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a1fa54-276a-4c67-825f-7067807bdb57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74fd797448f1479eb03bf66224f6d425)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:28.820561 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a1fa54-276a-4c67-825f-7067807bdb57 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:29.021447 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-42a1fa54-276a-4c67-825f-7067807bdb57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74fd797448f1479eb03bf66224f6d425'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:29.028771 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-42a1fa54-276a-4c67-825f-7067807bdb57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74fd797448f1479eb03bf66224f6d425'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:29.029976 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-42a1fa54-276a-4c67-825f-7067807bdb57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74fd797448f1479eb03bf66224f6d425'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:29.043635 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-42a1fa54-276a-4c67-825f-7067807bdb57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74fd797448f1479eb03bf66224f6d425'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:29.120558 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-42a1fa54-276a-4c67-825f-7067807bdb57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74fd797448f1479eb03bf66224f6d425'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:29.128048 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1220/2408] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:23:28 2022] DELETE /identity/v3/users/74fd797448f1479eb03bf66224f6d425 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:23:29.699877 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a38ac-1394-440d-93ca-c37adccdfa67 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:29.701173 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-0b1a38ac-1394-440d-93ca-c37adccdfa67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:29.716925 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a38ac-1394-440d-93ca-c37adccdfa67 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:29.785838 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:23:29.792519 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f9fd8-e13d-4055-ad6b-6e8ebdcc5c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:29.793423 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f9fd8-e13d-4055-ad6b-6e8ebdcc5c4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:29.794676 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f9fd8-e13d-4055-ad6b-6e8ebdcc5c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:29.795512 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f9fd8-e13d-4055-ad6b-6e8ebdcc5c4d admin admin] PATH_INFO: `/v3/projects/2cd7aebea8a34b2f847e8219762395ea` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:29.796708 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21f9fd8-e13d-4055-ad6b-6e8ebdcc5c4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cd7aebea8a34b2f847e8219762395ea)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:29.798627 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21f9fd8-e13d-4055-ad6b-6e8ebdcc5c4d admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:29.904543 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-e21f9fd8-e13d-4055-ad6b-6e8ebdcc5c4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cd7aebea8a34b2f847e8219762395ea', 'request_id': 'req-e21f9fd8-e13d-4055-ad6b-6e8ebdcc5c4d'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:29.910122 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-e21f9fd8-e13d-4055-ad6b-6e8ebdcc5c4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cd7aebea8a34b2f847e8219762395ea', 'request_id': 'req-e21f9fd8-e13d-4055-ad6b-6e8ebdcc5c4d'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:29.981405 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-e21f9fd8-e13d-4055-ad6b-6e8ebdcc5c4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cd7aebea8a34b2f847e8219762395ea', 'request_id': 'req-e21f9fd8-e13d-4055-ad6b-6e8ebdcc5c4d'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:29.996223 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-e21f9fd8-e13d-4055-ad6b-6e8ebdcc5c4d admin admin] The token cache is being invalidate because project 2cd7aebea8a34b2f847e8219762395ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:23:29.996522 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-e21f9fd8-e13d-4055-ad6b-6e8ebdcc5c4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:29.997266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-e21f9fd8-e13d-4055-ad6b-6e8ebdcc5c4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:23:30.008011 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1189/2409] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:23:29 2022] DELETE /identity/v3/projects/2cd7aebea8a34b2f847e8219762395ea => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:23:31.055312 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1fa54-276a-4c67-825f-7067807bdb57 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:31.056905 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-42a1fa54-276a-4c67-825f-7067807bdb57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:31.120391 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:23:31.137469 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1fa54-276a-4c67-825f-7067807bdb57 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:31.198451 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e893c55-39a4-445c-9648-15a94d49b58d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193637acadc14ec4a48ab79566ba3dab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:31.199065 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e893c55-39a4-445c-9648-15a94d49b58d service placement] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:31.199345 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e893c55-39a4-445c-9648-15a94d49b58d service placement] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:31.199722 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e893c55-39a4-445c-9648-15a94d49b58d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:31.202070 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-3e893c55-39a4-445c-9648-15a94d49b58d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:31.215226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e893c55-39a4-445c-9648-15a94d49b58d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:31.216940 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e893c55-39a4-445c-9648-15a94d49b58d service placement] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:31.229564 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1221/2410] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:23:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:23:40.047929 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=25,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:23:41.275364 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=32 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:23:46.001235 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f9fd8-e13d-4055-ad6b-6e8ebdcc5c4d admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:23:46.002644 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-e21f9fd8-e13d-4055-ad6b-6e8ebdcc5c4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:46.062369 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f9fd8-e13d-4055-ad6b-6e8ebdcc5c4d admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:23:46.066017 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c22726-e3d1-4474-9bef-c9eb34461714 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:23:46.066629 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c22726-e3d1-4474-9bef-c9eb34461714 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:23:46.066932 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c22726-e3d1-4474-9bef-c9eb34461714 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:23:46.067211 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c22726-e3d1-4474-9bef-c9eb34461714 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:23:46.069012 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-13c22726-e3d1-4474-9bef-c9eb34461714 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:23:46.148035 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c22726-e3d1-4474-9bef-c9eb34461714 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:23:46.148035 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c22726-e3d1-4474-9bef-c9eb34461714 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:23:46.149269 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1190/2411] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:23:46 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:23:56.144264 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=50 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:24:07.616938 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21190884-fe44-4924-9518-691b43c056bc None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:24:07.617469 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21190884-fe44-4924-9518-691b43c056bc None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:24:07.617469 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21190884-fe44-4924-9518-691b43c056bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:24:07.637091 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-21190884-fe44-4924-9518-691b43c056bc None None] MFA Rules not processed for user `e6b767c363bb4a6fa3530f6a677e09f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:24:07.638363 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-21190884-fe44-4924-9518-691b43c056bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:24:07.698188 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1222/2412] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:24:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:24:17.689081 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=25 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:24:22.028893 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62cce5-51f2-4d1f-bef2-4af9467aca0e None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:24:22.029390 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62cce5-51f2-4d1f-bef2-4af9467aca0e None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:24:22.029489 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62cce5-51f2-4d1f-bef2-4af9467aca0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:24:22.051874 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-5b62cce5-51f2-4d1f-bef2-4af9467aca0e None None] MFA Rules not processed for user `b6880278f7304051b9ef03e927637e74`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:24:22.053104 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-5b62cce5-51f2-4d1f-bef2-4af9467aca0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:24:22.108939 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1191/2413] 10.222.0.98 () {58 vars in 1009 bytes} [Thu Aug 18 12:24:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:24:22.154423 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21190884-fe44-4924-9518-691b43c056bc None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:24:22.156257 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-21190884-fe44-4924-9518-691b43c056bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:24:22.229517 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21190884-fe44-4924-9518-691b43c056bc None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:24:22.234764 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d248fae-93d5-4534-91f0-bc597b8016df service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771591fe31cb4e8c9c4320c2a7b2719d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:24:22.235382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d248fae-93d5-4534-91f0-bc597b8016df service swift] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:24:22.235719 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d248fae-93d5-4534-91f0-bc597b8016df service swift] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:24:22.236000 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d248fae-93d5-4534-91f0-bc597b8016df service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:24:22.246124 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d248fae-93d5-4534-91f0-bc597b8016df service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:24:22.247820 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d248fae-93d5-4534-91f0-bc597b8016df service swift] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:24:22.256414 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1223/2414] 10.222.0.98 () {60 vars in 1384 bytes} [Thu Aug 18 12:24:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 12:24:32.106786 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=25 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:24:32.253289 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=32 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:24:42.383035 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aca6d7-bda7-4f4a-83ad-e2d838323d8b None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:24:42.384382 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aca6d7-bda7-4f4a-83ad-e2d838323d8b None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:24:42.384697 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aca6d7-bda7-4f4a-83ad-e2d838323d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:24:42.405765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-d7aca6d7-bda7-4f4a-83ad-e2d838323d8b None None] MFA Rules not processed for user `2e8380a466484c83bca842518d171048`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:24:42.407068 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-d7aca6d7-bda7-4f4a-83ad-e2d838323d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:24:42.469649 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1192/2415] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:24:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:24:43.844332 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf13fc-4123-441d-99b9-daa4db319371 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:24:43.844827 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf13fc-4123-441d-99b9-daa4db319371 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:24:43.844926 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf13fc-4123-441d-99b9-daa4db319371 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:24:43.846237 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1224/2416] 10.222.0.98 () {54 vars in 921 bytes} [Thu Aug 18 12:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:24:43.853299 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09451f-b9d4-493f-896f-59249be88741 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:24:43.853877 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09451f-b9d4-493f-896f-59249be88741 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:24:43.854003 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09451f-b9d4-493f-896f-59249be88741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:24:43.877028 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-4a09451f-b9d4-493f-896f-59249be88741 None None] MFA Rules not processed for user `171c72a6b29e418bbb4ea5c6faffe3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:24:43.878621 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-4a09451f-b9d4-493f-896f-59249be88741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:24:43.956169 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1193/2417] 10.222.0.98 () {58 vars in 1018 bytes} [Thu Aug 18 12:24:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:24:43.968105 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf13fc-4123-441d-99b9-daa4db319371 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:24:43.969788 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-ccdf13fc-4123-441d-99b9-daa4db319371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:24:44.030742 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf13fc-4123-441d-99b9-daa4db319371 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:24:44.036756 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f5324-fc3f-4d86-94da-e66ef88619ae service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd020063702b0481e9f4111680766f7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:24:44.037717 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f5324-fc3f-4d86-94da-e66ef88619ae service glance] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:24:44.038155 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f5324-fc3f-4d86-94da-e66ef88619ae service glance] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:24:44.038507 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f5324-fc3f-4d86-94da-e66ef88619ae service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:24:44.050318 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014f5324-fc3f-4d86-94da-e66ef88619ae service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:24:44.051963 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014f5324-fc3f-4d86-94da-e66ef88619ae service glance] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:24:44.063160 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1225/2418] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:24:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:24:44.240440 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d90051-457d-48c3-9523-7c0ad1842510 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:24:44.240855 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d90051-457d-48c3-9523-7c0ad1842510 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:24:44.241239 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d90051-457d-48c3-9523-7c0ad1842510 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:24:44.242670 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1194/2419] 10.222.0.98 () {54 vars in 921 bytes} [Thu Aug 18 12:24:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:24:44.248325 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ff470-2693-4af8-904c-d370f3f2064e None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:24:44.248682 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ff470-2693-4af8-904c-d370f3f2064e None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:24:44.249112 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ff470-2693-4af8-904c-d370f3f2064e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:24:44.269838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-045ff470-2693-4af8-904c-d370f3f2064e None None] MFA Rules not processed for user `185644affb284c2b91e604cf99803ffb`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:24:44.271054 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-045ff470-2693-4af8-904c-d370f3f2064e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:24:44.277716 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1226/2420] 10.222.0.98 () {58 vars in 1018 bytes} [Thu Aug 18 12:24:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:24:44.289699 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d90051-457d-48c3-9523-7c0ad1842510 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:24:44.301901 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d90051-457d-48c3-9523-7c0ad1842510 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:24:44.305415 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de806e-5425-4848-b7b6-454a87166a89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:24:44.306015 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de806e-5425-4848-b7b6-454a87166a89 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:24:44.306292 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de806e-5425-4848-b7b6-454a87166a89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:24:44.306562 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de806e-5425-4848-b7b6-454a87166a89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:24:44.313969 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26de806e-5425-4848-b7b6-454a87166a89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:24:44.315385 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26de806e-5425-4848-b7b6-454a87166a89 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:24:44.323778 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1195/2421] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:24:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:24:44.408497 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7549f-9d35-4333-bf9e-66c25775d263 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:24:44.408787 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7549f-9d35-4333-bf9e-66c25775d263 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:24:44.409068 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7549f-9d35-4333-bf9e-66c25775d263 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:24:44.410516 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1227/2422] 10.222.0.98 () {54 vars in 921 bytes} [Thu Aug 18 12:24:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:24:44.416936 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73bbc5e-182e-4197-afd5-445e3ec46369 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:24:44.417229 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73bbc5e-182e-4197-afd5-445e3ec46369 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:24:44.417511 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73bbc5e-182e-4197-afd5-445e3ec46369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:24:44.461010 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-b73bbc5e-182e-4197-afd5-445e3ec46369 None None] MFA Rules not processed for user `171c72a6b29e418bbb4ea5c6faffe3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:24:44.462757 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-b73bbc5e-182e-4197-afd5-445e3ec46369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:24:44.473681 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1196/2423] 10.222.0.98 () {58 vars in 1018 bytes} [Thu Aug 18 12:24:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:24:44.489134 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc7549f-9d35-4333-bf9e-66c25775d263 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:24:44.500602 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc7549f-9d35-4333-bf9e-66c25775d263 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:24:44.504843 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5152ccb-277a-4044-a395-a63dc7b5eead service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:24:44.505461 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5152ccb-277a-4044-a395-a63dc7b5eead service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:24:44.505742 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5152ccb-277a-4044-a395-a63dc7b5eead service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:24:44.506309 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5152ccb-277a-4044-a395-a63dc7b5eead service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:24:44.523707 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5152ccb-277a-4044-a395-a63dc7b5eead service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:24:44.525188 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5152ccb-277a-4044-a395-a63dc7b5eead service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:24:44.539695 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1228/2424] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:24:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:24:52.124240 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0a721-efe5-42a0-a726-7e89d146cc31 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:24:52.124713 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0a721-efe5-42a0-a726-7e89d146cc31 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:24:52.124858 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0a721-efe5-42a0-a726-7e89d146cc31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:24:52.144576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-7de0a721-efe5-42a0-a726-7e89d146cc31 None None] MFA Rules not processed for user `b6880278f7304051b9ef03e927637e74`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:24:52.146436 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-7de0a721-efe5-42a0-a726-7e89d146cc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:24:52.153729 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1197/2425] 10.222.0.98 () {58 vars in 1009 bytes} [Thu Aug 18 12:24:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:24:52.165473 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5152ccb-277a-4044-a395-a63dc7b5eead service neutron] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:24:52.176901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5152ccb-277a-4044-a395-a63dc7b5eead service neutron] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:24:52.181911 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee3efc7-205a-47a2-af84-d01fce156f9a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771591fe31cb4e8c9c4320c2a7b2719d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:24:52.182499 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3efc7-205a-47a2-af84-d01fce156f9a service swift] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:24:52.182780 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3efc7-205a-47a2-af84-d01fce156f9a service swift] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:24:52.183069 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3efc7-205a-47a2-af84-d01fce156f9a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:24:52.191100 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee3efc7-205a-47a2-af84-d01fce156f9a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:24:52.193213 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee3efc7-205a-47a2-af84-d01fce156f9a service swift] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:24:52.205181 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1229/2426] 10.222.0.98 () {60 vars in 1384 bytes} [Thu Aug 18 12:24:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 12:25:02.139129 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=68 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:25:02.198778 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=41 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:25:15.022102 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f3966-e92a-454f-99f1-c28362569cb2 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:25:15.023029 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f3966-e92a-454f-99f1-c28362569cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:25:15.023503 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f3966-e92a-454f-99f1-c28362569cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:25:15.044322 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-617f3966-e92a-454f-99f1-c28362569cb2 None None] MFA Rules not processed for user `b6880278f7304051b9ef03e927637e74`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:25:15.045581 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-617f3966-e92a-454f-99f1-c28362569cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:25:15.053117 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1198/2427] 10.222.0.98 () {58 vars in 967 bytes} [Thu Aug 18 12:25:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:25:15.066779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee3efc7-205a-47a2-af84-d01fce156f9a service swift] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:25:15.078105 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee3efc7-205a-47a2-af84-d01fce156f9a service swift] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:25:15.083369 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d010846-8e8e-4541-9b77-0ba15762ee7c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771591fe31cb4e8c9c4320c2a7b2719d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:25:15.084075 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d010846-8e8e-4541-9b77-0ba15762ee7c service swift] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:25:15.084363 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d010846-8e8e-4541-9b77-0ba15762ee7c service swift] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:25:15.084647 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d010846-8e8e-4541-9b77-0ba15762ee7c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:25:15.093595 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d010846-8e8e-4541-9b77-0ba15762ee7c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:25:15.095207 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d010846-8e8e-4541-9b77-0ba15762ee7c service swift] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:25:15.103580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1230/2428] 10.222.0.98 () {60 vars in 1384 bytes} [Thu Aug 18 12:25:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 12:25:15.841843 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025d6fa-4591-4efc-ae1c-d16f12bf37be None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:25:15.842137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025d6fa-4591-4efc-ae1c-d16f12bf37be None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:25:15.842421 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025d6fa-4591-4efc-ae1c-d16f12bf37be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:25:15.861749 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-a025d6fa-4591-4efc-ae1c-d16f12bf37be None None] MFA Rules not processed for user `b5113b1a77fd4b5da175b29b40d2b636`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:25:15.862846 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-a025d6fa-4591-4efc-ae1c-d16f12bf37be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:25:15.920741 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1199/2429] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:25:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:25:17.827207 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2817d-943a-425e-bce9-145a2403c2f4 None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:25:17.827759 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2817d-943a-425e-bce9-145a2403c2f4 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:25:17.828724 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2817d-943a-425e-bce9-145a2403c2f4 None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:25:17.829382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1231/2430] 10.222.0.98 () {54 vars in 921 bytes} [Thu Aug 18 12:25:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:25:17.844358 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a7e2c-dbb3-4ccd-92b6-e85a4579b315 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:25:17.848295 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a7e2c-dbb3-4ccd-92b6-e85a4579b315 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:25:17.848295 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a7e2c-dbb3-4ccd-92b6-e85a4579b315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:25:17.866779 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-6d8a7e2c-dbb3-4ccd-92b6-e85a4579b315 None None] MFA Rules not processed for user `193637acadc14ec4a48ab79566ba3dab`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:25:17.868151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-6d8a7e2c-dbb3-4ccd-92b6-e85a4579b315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:25:17.878796 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1200/2431] 10.222.0.98 () {58 vars in 1018 bytes} [Thu Aug 18 12:25:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:25:17.896148 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2817d-943a-425e-bce9-145a2403c2f4 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:25:17.898034 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-12f2817d-943a-425e-bce9-145a2403c2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:25:17.914852 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2817d-943a-425e-bce9-145a2403c2f4 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:25:17.920505 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f1df0f-2cd4-45bb-b154-e3e9a00ec0af service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193637acadc14ec4a48ab79566ba3dab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:25:17.921131 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f1df0f-2cd4-45bb-b154-e3e9a00ec0af service placement] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:25:17.921399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f1df0f-2cd4-45bb-b154-e3e9a00ec0af service placement] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:25:17.921680 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f1df0f-2cd4-45bb-b154-e3e9a00ec0af service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:25:17.930334 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f1df0f-2cd4-45bb-b154-e3e9a00ec0af service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:25:17.931892 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f1df0f-2cd4-45bb-b154-e3e9a00ec0af service placement] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:25:17.942593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1232/2432] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:25:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:25:18.017873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a7e2c-dbb3-4ccd-92b6-e85a4579b315 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:25:18.020299 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-6d8a7e2c-dbb3-4ccd-92b6-e85a4579b315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:25:18.095280 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a7e2c-dbb3-4ccd-92b6-e85a4579b315 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:25:18.099917 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c3aaf-90ac-4734-8e28-e053d6abd552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:25:18.100547 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c3aaf-90ac-4734-8e28-e053d6abd552 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:25:18.100826 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c3aaf-90ac-4734-8e28-e053d6abd552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:25:18.101097 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c3aaf-90ac-4734-8e28-e053d6abd552 admin admin] PATH_INFO: `/v3/users/e6b767c363bb4a6fa3530f6a677e09f2` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:25:18.102304 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c3aaf-90ac-4734-8e28-e053d6abd552 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6b767c363bb4a6fa3530f6a677e09f2)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:25:18.104266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c3aaf-90ac-4734-8e28-e053d6abd552 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:25:18.289161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-481c3aaf-90ac-4734-8e28-e053d6abd552 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6b767c363bb4a6fa3530f6a677e09f2'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:18.289738 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-481c3aaf-90ac-4734-8e28-e053d6abd552 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6b767c363bb4a6fa3530f6a677e09f2'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:18.290340 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-481c3aaf-90ac-4734-8e28-e053d6abd552 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6b767c363bb4a6fa3530f6a677e09f2'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:18.297772 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-481c3aaf-90ac-4734-8e28-e053d6abd552 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6b767c363bb4a6fa3530f6a677e09f2'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:18.309878 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-481c3aaf-90ac-4734-8e28-e053d6abd552 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6b767c363bb4a6fa3530f6a677e09f2'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:18.407034 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1201/2433] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:25:18 2022] DELETE /identity/v3/users/e6b767c363bb4a6fa3530f6a677e09f2 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:25:18.671568 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f1df0f-2cd4-45bb-b154-e3e9a00ec0af service placement] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:25:18.682579 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f1df0f-2cd4-45bb-b154-e3e9a00ec0af service placement] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:25:18.750679 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3589686f-dbf4-4802-9991-ccaf3dcb7730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:25:18.751319 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589686f-dbf4-4802-9991-ccaf3dcb7730 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:25:18.751647 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589686f-dbf4-4802-9991-ccaf3dcb7730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:25:18.751931 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589686f-dbf4-4802-9991-ccaf3dcb7730 admin admin] PATH_INFO: `/v3/projects/c01ab18435804769bb4ae0bf86d4d99d` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:25:18.753219 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3589686f-dbf4-4802-9991-ccaf3dcb7730 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c01ab18435804769bb4ae0bf86d4d99d)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:25:18.755151 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3589686f-dbf4-4802-9991-ccaf3dcb7730 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:25:18.888184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-3589686f-dbf4-4802-9991-ccaf3dcb7730 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c01ab18435804769bb4ae0bf86d4d99d', 'request_id': 'req-3589686f-dbf4-4802-9991-ccaf3dcb7730'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:18.894093 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-3589686f-dbf4-4802-9991-ccaf3dcb7730 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c01ab18435804769bb4ae0bf86d4d99d', 'request_id': 'req-3589686f-dbf4-4802-9991-ccaf3dcb7730'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:18.994833 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-3589686f-dbf4-4802-9991-ccaf3dcb7730 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c01ab18435804769bb4ae0bf86d4d99d', 'request_id': 'req-3589686f-dbf4-4802-9991-ccaf3dcb7730'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:19.008186 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-3589686f-dbf4-4802-9991-ccaf3dcb7730 admin admin] The token cache is being invalidate because project c01ab18435804769bb4ae0bf86d4d99d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:25:19.008477 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-3589686f-dbf4-4802-9991-ccaf3dcb7730 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:19.009145 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-3589686f-dbf4-4802-9991-ccaf3dcb7730 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:19.010709 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1233/2434] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:25:18 2022] DELETE /identity/v3/projects/c01ab18435804769bb4ae0bf86d4d99d => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:25:20.272294 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c3aaf-90ac-4734-8e28-e053d6abd552 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:25:20.273778 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-481c3aaf-90ac-4734-8e28-e053d6abd552 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:25:20.287825 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:25:20.333165 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c3aaf-90ac-4734-8e28-e053d6abd552 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:25:20.385153 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184ef64-234e-463e-b522-10165e0a5c16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:25:20.385761 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184ef64-234e-463e-b522-10165e0a5c16 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:25:20.386041 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184ef64-234e-463e-b522-10165e0a5c16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:25:20.386313 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184ef64-234e-463e-b522-10165e0a5c16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:25:20.388774 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-3184ef64-234e-463e-b522-10165e0a5c16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:25:20.464270 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184ef64-234e-463e-b522-10165e0a5c16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:25:20.465768 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184ef64-234e-463e-b522-10165e0a5c16 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:25:20.477702 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1202/2435] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:25:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:25:22.880942 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3589686f-dbf4-4802-9991-ccaf3dcb7730 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:25:22.882450 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-3589686f-dbf4-4802-9991-ccaf3dcb7730 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:25:22.895017 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3589686f-dbf4-4802-9991-ccaf3dcb7730 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:25:22.898965 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0972bc06-fb17-4f15-9c3b-eab60ac5562d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:25:22.899602 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972bc06-fb17-4f15-9c3b-eab60ac5562d service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:25:22.903447 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972bc06-fb17-4f15-9c3b-eab60ac5562d service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:25:22.903447 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972bc06-fb17-4f15-9c3b-eab60ac5562d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:25:22.903447 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-0972bc06-fb17-4f15-9c3b-eab60ac5562d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:25:22.917477 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0972bc06-fb17-4f15-9c3b-eab60ac5562d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:25:22.918949 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0972bc06-fb17-4f15-9c3b-eab60ac5562d service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:25:22.930977 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1234/2436] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:25:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:25:30.503907 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=69 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:25:32.703735 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184ef64-234e-463e-b522-10165e0a5c16 service neutron] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:25:32.705147 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-3184ef64-234e-463e-b522-10165e0a5c16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:25:32.719070 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184ef64-234e-463e-b522-10165e0a5c16 service neutron] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:25:32.725689 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64991173-bc07-4c47-babc-075f410725ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:25:32.726296 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64991173-bc07-4c47-babc-075f410725ab service nova] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:25:32.726576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64991173-bc07-4c47-babc-075f410725ab service nova] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:25:32.726848 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64991173-bc07-4c47-babc-075f410725ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:25:32.728609 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-64991173-bc07-4c47-babc-075f410725ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:25:32.789311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64991173-bc07-4c47-babc-075f410725ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:25:32.790815 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64991173-bc07-4c47-babc-075f410725ab service nova] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:25:32.800739 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1203/2437] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:25:32 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:25:32.925978 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=69,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:25:38.139219 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a405f30-8bb5-4b39-a202-a3088f8ba87d None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:25:38.139888 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a405f30-8bb5-4b39-a202-a3088f8ba87d None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:25:38.139888 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a405f30-8bb5-4b39-a202-a3088f8ba87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:25:38.159346 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-4a405f30-8bb5-4b39-a202-a3088f8ba87d None None] MFA Rules not processed for user `b6880278f7304051b9ef03e927637e74`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:25:38.160724 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-4a405f30-8bb5-4b39-a202-a3088f8ba87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:25:38.219235 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1235/2438] 10.222.0.98 () {58 vars in 967 bytes} [Thu Aug 18 12:25:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:25:38.245347 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64991173-bc07-4c47-babc-075f410725ab service nova] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:25:38.247041 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-64991173-bc07-4c47-babc-075f410725ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:25:38.323774 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64991173-bc07-4c47-babc-075f410725ab service nova] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:25:38.328906 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a7c102-7e2f-45f3-bafb-4cf7b87ceb71 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771591fe31cb4e8c9c4320c2a7b2719d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:25:38.329547 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a7c102-7e2f-45f3-bafb-4cf7b87ceb71 service swift] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:25:38.329826 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a7c102-7e2f-45f3-bafb-4cf7b87ceb71 service swift] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:25:38.330098 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a7c102-7e2f-45f3-bafb-4cf7b87ceb71 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:25:38.338709 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a7c102-7e2f-45f3-bafb-4cf7b87ceb71 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:25:38.340394 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a7c102-7e2f-45f3-bafb-4cf7b87ceb71 service swift] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:25:38.349514 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1204/2439] 10.222.0.98 () {60 vars in 1384 bytes} [Thu Aug 18 12:25:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 12:25:47.149979 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a405f30-8bb5-4b39-a202-a3088f8ba87d None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:25:47.151587 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-4a405f30-8bb5-4b39-a202-a3088f8ba87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:25:47.225359 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a405f30-8bb5-4b39-a202-a3088f8ba87d None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:25:47.230335 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28448b41-70a9-491d-b9f0-6e28d1ffff19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:25:47.230984 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28448b41-70a9-491d-b9f0-6e28d1ffff19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:25:47.231354 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28448b41-70a9-491d-b9f0-6e28d1ffff19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:25:47.231684 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28448b41-70a9-491d-b9f0-6e28d1ffff19 admin admin] PATH_INFO: `/v3/users/fcc7f1b599034654a3ed64dde2145bf8` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:25:47.233008 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28448b41-70a9-491d-b9f0-6e28d1ffff19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcc7f1b599034654a3ed64dde2145bf8)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:25:47.235736 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28448b41-70a9-491d-b9f0-6e28d1ffff19 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:25:47.438183 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-28448b41-70a9-491d-b9f0-6e28d1ffff19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcc7f1b599034654a3ed64dde2145bf8'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:47.445852 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-28448b41-70a9-491d-b9f0-6e28d1ffff19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcc7f1b599034654a3ed64dde2145bf8'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:47.446548 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-28448b41-70a9-491d-b9f0-6e28d1ffff19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcc7f1b599034654a3ed64dde2145bf8'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:47.460568 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-28448b41-70a9-491d-b9f0-6e28d1ffff19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcc7f1b599034654a3ed64dde2145bf8'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:47.507909 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-28448b41-70a9-491d-b9f0-6e28d1ffff19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcc7f1b599034654a3ed64dde2145bf8'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:47.514404 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1236/2440] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:25:47 2022] DELETE /identity/v3/users/fcc7f1b599034654a3ed64dde2145bf8 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:25:47.772671 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a7c102-7e2f-45f3-bafb-4cf7b87ceb71 service swift] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:25:47.785142 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a7c102-7e2f-45f3-bafb-4cf7b87ceb71 service swift] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:25:47.842577 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8b771-4559-4121-9a93-48d82c7e2441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:25:47.843489 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8b771-4559-4121-9a93-48d82c7e2441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:25:47.843489 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8b771-4559-4121-9a93-48d82c7e2441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:25:47.843869 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8b771-4559-4121-9a93-48d82c7e2441 admin admin] PATH_INFO: `/v3/projects/944cd11144ca4cbf86962a3c9494ecbd` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:25:47.845653 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc8b771-4559-4121-9a93-48d82c7e2441 admin admin] RBAC: Authorizing `identity:delete_project(project_id=944cd11144ca4cbf86962a3c9494ecbd)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:25:47.847595 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc8b771-4559-4121-9a93-48d82c7e2441 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:25:47.975260 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:25:47.988801 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-6cc8b771-4559-4121-9a93-48d82c7e2441 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '944cd11144ca4cbf86962a3c9494ecbd', 'request_id': 'req-6cc8b771-4559-4121-9a93-48d82c7e2441'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:47.994214 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-6cc8b771-4559-4121-9a93-48d82c7e2441 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '944cd11144ca4cbf86962a3c9494ecbd', 'request_id': 'req-6cc8b771-4559-4121-9a93-48d82c7e2441'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:48.079709 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-6cc8b771-4559-4121-9a93-48d82c7e2441 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '944cd11144ca4cbf86962a3c9494ecbd', 'request_id': 'req-6cc8b771-4559-4121-9a93-48d82c7e2441'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:48.092544 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-6cc8b771-4559-4121-9a93-48d82c7e2441 admin admin] The token cache is being invalidate because project 944cd11144ca4cbf86962a3c9494ecbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:25:48.093256 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-6cc8b771-4559-4121-9a93-48d82c7e2441 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:48.094437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-6cc8b771-4559-4121-9a93-48d82c7e2441 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:48.096129 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1205/2441] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:25:47 2022] DELETE /identity/v3/projects/944cd11144ca4cbf86962a3c9494ecbd => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:25:48.100382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28448b41-70a9-491d-b9f0-6e28d1ffff19 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:25:48.101797 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-28448b41-70a9-491d-b9f0-6e28d1ffff19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:25:48.115871 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28448b41-70a9-491d-b9f0-6e28d1ffff19 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:25:48.135292 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:25:48.178841 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7bf9e-d557-41a6-98c3-cde02ca998ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:25:48.178841 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7bf9e-d557-41a6-98c3-cde02ca998ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:25:48.178841 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7bf9e-d557-41a6-98c3-cde02ca998ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:25:48.178841 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7bf9e-d557-41a6-98c3-cde02ca998ff admin admin] PATH_INFO: `/v3/users/d26989d779e645159c80b444394c2e66` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:25:48.181518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b7bf9e-d557-41a6-98c3-cde02ca998ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=d26989d779e645159c80b444394c2e66)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:25:48.181518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b7bf9e-d557-41a6-98c3-cde02ca998ff admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:25:48.396752 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-12b7bf9e-d557-41a6-98c3-cde02ca998ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd26989d779e645159c80b444394c2e66'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:48.403621 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-12b7bf9e-d557-41a6-98c3-cde02ca998ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd26989d779e645159c80b444394c2e66'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:48.404367 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-12b7bf9e-d557-41a6-98c3-cde02ca998ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd26989d779e645159c80b444394c2e66'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:48.416890 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-12b7bf9e-d557-41a6-98c3-cde02ca998ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd26989d779e645159c80b444394c2e66'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:48.518847 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-12b7bf9e-d557-41a6-98c3-cde02ca998ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd26989d779e645159c80b444394c2e66'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:48.525257 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1237/2442] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:25:48 2022] DELETE /identity/v3/users/d26989d779e645159c80b444394c2e66 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:25:48.790018 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8b771-4559-4121-9a93-48d82c7e2441 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:25:48.791550 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-6cc8b771-4559-4121-9a93-48d82c7e2441 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:25:48.805160 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8b771-4559-4121-9a93-48d82c7e2441 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:25:48.865539 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33d8658-bb74-47f4-9dc1-dd8933405649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:25:48.866174 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d8658-bb74-47f4-9dc1-dd8933405649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:25:48.866458 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d8658-bb74-47f4-9dc1-dd8933405649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:25:48.866743 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d8658-bb74-47f4-9dc1-dd8933405649 admin admin] PATH_INFO: `/v3/projects/1ee562f56a3b494e8c8edc09cb945508` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:25:48.867942 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33d8658-bb74-47f4-9dc1-dd8933405649 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ee562f56a3b494e8c8edc09cb945508)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:25:48.869854 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33d8658-bb74-47f4-9dc1-dd8933405649 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:25:48.972633 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-a33d8658-bb74-47f4-9dc1-dd8933405649 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ee562f56a3b494e8c8edc09cb945508', 'request_id': 'req-a33d8658-bb74-47f4-9dc1-dd8933405649'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:48.978098 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-a33d8658-bb74-47f4-9dc1-dd8933405649 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ee562f56a3b494e8c8edc09cb945508', 'request_id': 'req-a33d8658-bb74-47f4-9dc1-dd8933405649'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:49.050403 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-a33d8658-bb74-47f4-9dc1-dd8933405649 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ee562f56a3b494e8c8edc09cb945508', 'request_id': 'req-a33d8658-bb74-47f4-9dc1-dd8933405649'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:49.062291 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-a33d8658-bb74-47f4-9dc1-dd8933405649 admin admin] The token cache is being invalidate because project 1ee562f56a3b494e8c8edc09cb945508 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:25:49.062612 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-a33d8658-bb74-47f4-9dc1-dd8933405649 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:49.063314 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-a33d8658-bb74-47f4-9dc1-dd8933405649 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:49.064836 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1206/2443] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:25:48 2022] DELETE /identity/v3/projects/1ee562f56a3b494e8c8edc09cb945508 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:25:53.343129 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7bf9e-d557-41a6-98c3-cde02ca998ff admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:25:53.345112 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-12b7bf9e-d557-41a6-98c3-cde02ca998ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:25:53.407454 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7bf9e-d557-41a6-98c3-cde02ca998ff admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:25:53.456375 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dddf2d-7ab6-4cf5-afef-acf13cb61083 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:25:53.456988 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dddf2d-7ab6-4cf5-afef-acf13cb61083 service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:25:53.457274 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dddf2d-7ab6-4cf5-afef-acf13cb61083 service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:25:53.457552 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dddf2d-7ab6-4cf5-afef-acf13cb61083 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:25:53.459325 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-23dddf2d-7ab6-4cf5-afef-acf13cb61083 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:25:53.471984 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:25:53.522111 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dddf2d-7ab6-4cf5-afef-acf13cb61083 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:25:53.523562 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dddf2d-7ab6-4cf5-afef-acf13cb61083 service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:25:53.533927 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1238/2444] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:25:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:25:57.189074 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33d8658-bb74-47f4-9dc1-dd8933405649 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:25:57.190520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-a33d8658-bb74-47f4-9dc1-dd8933405649 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:25:57.208714 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33d8658-bb74-47f4-9dc1-dd8933405649 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:25:57.215740 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c02637-c57f-4f63-b3a9-0ec932c2ffaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:25:57.217103 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c02637-c57f-4f63-b3a9-0ec932c2ffaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:25:57.217572 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c02637-c57f-4f63-b3a9-0ec932c2ffaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:25:57.218038 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c02637-c57f-4f63-b3a9-0ec932c2ffaa admin admin] PATH_INFO: `/v3/users/2e8380a466484c83bca842518d171048` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:25:57.219853 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c02637-c57f-4f63-b3a9-0ec932c2ffaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e8380a466484c83bca842518d171048)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:25:57.221770 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c02637-c57f-4f63-b3a9-0ec932c2ffaa admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:25:57.434949 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:25:57.437879 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-c1c02637-c57f-4f63-b3a9-0ec932c2ffaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e8380a466484c83bca842518d171048'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:57.438713 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-c1c02637-c57f-4f63-b3a9-0ec932c2ffaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e8380a466484c83bca842518d171048'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:57.439955 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-c1c02637-c57f-4f63-b3a9-0ec932c2ffaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e8380a466484c83bca842518d171048'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:57.447268 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-c1c02637-c57f-4f63-b3a9-0ec932c2ffaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e8380a466484c83bca842518d171048'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:57.460127 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-c1c02637-c57f-4f63-b3a9-0ec932c2ffaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e8380a466484c83bca842518d171048'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:57.525218 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1207/2445] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:25:57 2022] DELETE /identity/v3/users/2e8380a466484c83bca842518d171048 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:25:57.773682 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dddf2d-7ab6-4cf5-afef-acf13cb61083 service neutron] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:25:57.784191 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dddf2d-7ab6-4cf5-afef-acf13cb61083 service neutron] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:25:57.841983 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f0b9a-2b48-420a-a5f2-47a976b1a78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:25:57.842626 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f0b9a-2b48-420a-a5f2-47a976b1a78b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:25:57.842908 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f0b9a-2b48-420a-a5f2-47a976b1a78b admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:25:57.843187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f0b9a-2b48-420a-a5f2-47a976b1a78b admin admin] PATH_INFO: `/v3/projects/965cf5f488364e0ea8b52cdc78de2641` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:25:57.844409 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f0b9a-2b48-420a-a5f2-47a976b1a78b admin admin] RBAC: Authorizing `identity:delete_project(project_id=965cf5f488364e0ea8b52cdc78de2641)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:25:57.846334 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f0b9a-2b48-420a-a5f2-47a976b1a78b admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:25:57.959987 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-bd3f0b9a-2b48-420a-a5f2-47a976b1a78b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '965cf5f488364e0ea8b52cdc78de2641', 'request_id': 'req-bd3f0b9a-2b48-420a-a5f2-47a976b1a78b'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:57.965486 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-bd3f0b9a-2b48-420a-a5f2-47a976b1a78b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '965cf5f488364e0ea8b52cdc78de2641', 'request_id': 'req-bd3f0b9a-2b48-420a-a5f2-47a976b1a78b'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:58.142232 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-bd3f0b9a-2b48-420a-a5f2-47a976b1a78b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '965cf5f488364e0ea8b52cdc78de2641', 'request_id': 'req-bd3f0b9a-2b48-420a-a5f2-47a976b1a78b'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:58.154412 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-bd3f0b9a-2b48-420a-a5f2-47a976b1a78b admin admin] The token cache is being invalidate because project 965cf5f488364e0ea8b52cdc78de2641 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:25:58.154726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-bd3f0b9a-2b48-420a-a5f2-47a976b1a78b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:58.155375 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-bd3f0b9a-2b48-420a-a5f2-47a976b1a78b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:25:58.156894 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1239/2446] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:25:57 2022] DELETE /identity/v3/projects/965cf5f488364e0ea8b52cdc78de2641 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:25:58.161974 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a879434-b6ef-4bf5-9d09-f4d7ea188d8f None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:25:58.162281 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a879434-b6ef-4bf5-9d09-f4d7ea188d8f None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:25:58.162609 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a879434-b6ef-4bf5-9d09-f4d7ea188d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:25:58.184957 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-3a879434-b6ef-4bf5-9d09-f4d7ea188d8f None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:25:58.186189 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-3a879434-b6ef-4bf5-9d09-f4d7ea188d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:25:58.247934 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1208/2447] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:25:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:25:58.256016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647165c-1e06-474d-bca2-a06659267e51 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:25:58.256312 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647165c-1e06-474d-bca2-a06659267e51 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:25:58.256598 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647165c-1e06-474d-bca2-a06659267e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:25:58.276333 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-5647165c-1e06-474d-bca2-a06659267e51 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:25:58.277571 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-5647165c-1e06-474d-bca2-a06659267e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:25:58.285524 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1240/2448] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:25:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:25:58.289761 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a879434-b6ef-4bf5-9d09-f4d7ea188d8f None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:25:58.299891 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a879434-b6ef-4bf5-9d09-f4d7ea188d8f None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:25:58.304737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94ea1c-d30d-4c09-80f3-232dcfda937b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:25:58.305346 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94ea1c-d30d-4c09-80f3-232dcfda937b admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:25:58.305626 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94ea1c-d30d-4c09-80f3-232dcfda937b admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:25:58.305899 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94ea1c-d30d-4c09-80f3-232dcfda937b admin admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:25:58.306573 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d94ea1c-d30d-4c09-80f3-232dcfda937b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:25:58.306903 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d94ea1c-d30d-4c09-80f3-232dcfda937b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:25:58.308534 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d94ea1c-d30d-4c09-80f3-232dcfda937b admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:25:58.320808 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1209/2449] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:25:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:25:58.325005 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5647165c-1e06-474d-bca2-a06659267e51 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:25:58.334480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5647165c-1e06-474d-bca2-a06659267e51 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:25:58.339209 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e339d19d-ba61-4aee-bcad-91fb84145148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:25:58.339843 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339d19d-ba61-4aee-bcad-91fb84145148 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:25:58.340124 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339d19d-ba61-4aee-bcad-91fb84145148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:25:58.340400 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339d19d-ba61-4aee-bcad-91fb84145148 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:25:58.341028 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e339d19d-ba61-4aee-bcad-91fb84145148 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:25:58.342942 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e339d19d-ba61-4aee-bcad-91fb84145148 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:25:58.430099 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1241/2450] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:25:58 2022] POST /identity/v3/projects => generated 365 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:25:58.434305 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94ea1c-d30d-4c09-80f3-232dcfda937b admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:25:58.445054 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94ea1c-d30d-4c09-80f3-232dcfda937b admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:25:58.507609 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab0617e-ce58-49fa-8a88-1af4923ed92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:25:58.508226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab0617e-ce58-49fa-8a88-1af4923ed92b admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:25:58.509004 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab0617e-ce58-49fa-8a88-1af4923ed92b admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:25:58.509004 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab0617e-ce58-49fa-8a88-1af4923ed92b admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:25:58.509420 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab0617e-ce58-49fa-8a88-1af4923ed92b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:25:58.511362 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab0617e-ce58-49fa-8a88-1af4923ed92b admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:25:58.581264 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1210/2451] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:25:58 2022] POST /identity/v3/users => generated 283 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:25:58.582545 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e339d19d-ba61-4aee-bcad-91fb84145148 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:25:58.594704 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e339d19d-ba61-4aee-bcad-91fb84145148 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:25:58.601351 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53c386-93a2-4ab7-943c-9d122d7dc4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:25:58.601983 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53c386-93a2-4ab7-943c-9d122d7dc4ff admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:25:58.602270 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53c386-93a2-4ab7-943c-9d122d7dc4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:25:58.602653 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53c386-93a2-4ab7-943c-9d122d7dc4ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:25:58.603260 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f53c386-93a2-4ab7-943c-9d122d7dc4ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:25:58.605053 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f53c386-93a2-4ab7-943c-9d122d7dc4ff admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:25:58.612283 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:25:58.620582 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1242/2452] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:25:58.622503 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab0617e-ce58-49fa-8a88-1af4923ed92b admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:25:58.640540 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab0617e-ce58-49fa-8a88-1af4923ed92b admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:25:58.648409 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27993a6-25a1-4e16-89c5-abe845b6a22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:25:58.651694 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27993a6-25a1-4e16-89c5-abe845b6a22e admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:25:58.651694 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27993a6-25a1-4e16-89c5-abe845b6a22e admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:25:58.651694 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27993a6-25a1-4e16-89c5-abe845b6a22e admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:25:58.651694 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27993a6-25a1-4e16-89c5-abe845b6a22e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:25:58.654468 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27993a6-25a1-4e16-89c5-abe845b6a22e admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:25:58.676196 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1211/2453] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:25:58.682134 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53c386-93a2-4ab7-943c-9d122d7dc4ff admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:25:58.688419 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53c386-93a2-4ab7-943c-9d122d7dc4ff admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:25:58.693411 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f127eccc-4332-47dc-ae5f-033ecee9d76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:25:58.694121 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127eccc-4332-47dc-ae5f-033ecee9d76b admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:25:58.694369 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127eccc-4332-47dc-ae5f-033ecee9d76b admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:25:58.694650 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127eccc-4332-47dc-ae5f-033ecee9d76b admin admin] PATH_INFO: `/v3/projects/a6a1abc3ddbc44c782b8f130e6078018/users/7d9b49e8a6aa46cead8969f4225f6153/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:25:58.710927 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f127eccc-4332-47dc-ae5f-033ecee9d76b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6a1abc3ddbc44c782b8f130e6078018, user_id=7d9b49e8a6aa46cead8969f4225f6153, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:25:58.714715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f127eccc-4332-47dc-ae5f-033ecee9d76b admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:25:58.794425 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1243/2454] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:25:58 2022] PUT /identity/v3/projects/a6a1abc3ddbc44c782b8f130e6078018/users/7d9b49e8a6aa46cead8969f4225f6153/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:25:58.805447 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59558c-be31-47f8-9e43-bdb91e126071 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:25:58.806205 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59558c-be31-47f8-9e43-bdb91e126071 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:25:58.806205 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59558c-be31-47f8-9e43-bdb91e126071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:25:58.835611 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-8d59558c-be31-47f8-9e43-bdb91e126071 None None] MFA Rules not processed for user `7d9b49e8a6aa46cead8969f4225f6153`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:25:58.836946 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-8d59558c-be31-47f8-9e43-bdb91e126071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:25:58.851884 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:25:58.912455 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1212/2455] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:25:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:25:58.919534 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f127eccc-4332-47dc-ae5f-033ecee9d76b admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:25:58.991947 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f127eccc-4332-47dc-ae5f-033ecee9d76b admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:25:58.997826 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c721abe-6279-4d1e-a7e6-ac39f492fd93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:25:58.998537 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c721abe-6279-4d1e-a7e6-ac39f492fd93 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:25:58.998876 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c721abe-6279-4d1e-a7e6-ac39f492fd93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:25:58.999167 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c721abe-6279-4d1e-a7e6-ac39f492fd93 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:25:58.999977 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c721abe-6279-4d1e-a7e6-ac39f492fd93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:25:59.001969 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c721abe-6279-4d1e-a7e6-ac39f492fd93 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:25:59.132323 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1244/2456] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:25:58 2022] POST /identity/v3/projects => generated 363 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:25:59.136219 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d59558c-be31-47f8-9e43-bdb91e126071 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:25:59.149290 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d59558c-be31-47f8-9e43-bdb91e126071 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:25:59.213136 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9239ed7-ecad-4376-9078-6c8eee2f2488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:25:59.213956 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9239ed7-ecad-4376-9078-6c8eee2f2488 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:25:59.214250 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9239ed7-ecad-4376-9078-6c8eee2f2488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:25:59.214677 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9239ed7-ecad-4376-9078-6c8eee2f2488 admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:25:59.215374 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9239ed7-ecad-4376-9078-6c8eee2f2488 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:25:59.217368 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9239ed7-ecad-4376-9078-6c8eee2f2488 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:25:59.319189 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1213/2457] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:25:59 2022] POST /identity/v3/users => generated 288 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:25:59.323325 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c721abe-6279-4d1e-a7e6-ac39f492fd93 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:25:59.335042 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c721abe-6279-4d1e-a7e6-ac39f492fd93 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:25:59.340234 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b939e9-3bb5-4b18-8891-b59ce75b9bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:25:59.340843 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b939e9-3bb5-4b18-8891-b59ce75b9bef admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:25:59.341123 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b939e9-3bb5-4b18-8891-b59ce75b9bef admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:25:59.341397 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b939e9-3bb5-4b18-8891-b59ce75b9bef admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:25:59.342049 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b939e9-3bb5-4b18-8891-b59ce75b9bef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:25:59.343903 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b939e9-3bb5-4b18-8891-b59ce75b9bef admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:25:59.357140 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1245/2458] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:25:59.361940 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9239ed7-ecad-4376-9078-6c8eee2f2488 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:25:59.372252 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9239ed7-ecad-4376-9078-6c8eee2f2488 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:25:59.377656 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275a326f-cdda-4d50-a97e-b751a8e71dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:25:59.378322 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a326f-cdda-4d50-a97e-b751a8e71dd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:25:59.378601 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a326f-cdda-4d50-a97e-b751a8e71dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:25:59.378877 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a326f-cdda-4d50-a97e-b751a8e71dd1 admin admin] PATH_INFO: `/v3/projects/d8b5b40a32894fccb10722c58e023cbe/users/f913685133fa440d8ee08cbaa2fe43a0/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:25:59.400103 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275a326f-cdda-4d50-a97e-b751a8e71dd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8b5b40a32894fccb10722c58e023cbe, user_id=f913685133fa440d8ee08cbaa2fe43a0, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:25:59.402673 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275a326f-cdda-4d50-a97e-b751a8e71dd1 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:25:59.496238 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1214/2459] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:25:59 2022] PUT /identity/v3/projects/d8b5b40a32894fccb10722c58e023cbe/users/f913685133fa440d8ee08cbaa2fe43a0/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:25:59.500957 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b939e9-3bb5-4b18-8891-b59ce75b9bef admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:25:59.511374 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b939e9-3bb5-4b18-8891-b59ce75b9bef admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:25:59.570530 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e12815-e084-424a-9b18-6c3080af4585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:25:59.571149 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e12815-e084-424a-9b18-6c3080af4585 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:25:59.571463 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e12815-e084-424a-9b18-6c3080af4585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:25:59.571752 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e12815-e084-424a-9b18-6c3080af4585 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:25:59.572399 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e12815-e084-424a-9b18-6c3080af4585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:25:59.574041 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e12815-e084-424a-9b18-6c3080af4585 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:25:59.585540 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1246/2460] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:25:59.590586 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275a326f-cdda-4d50-a97e-b751a8e71dd1 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:25:59.600976 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275a326f-cdda-4d50-a97e-b751a8e71dd1 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:25:59.605890 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05316851-eb8f-48df-b06d-c04c05fa0681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:25:59.615935 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05316851-eb8f-48df-b06d-c04c05fa0681 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:25:59.615935 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05316851-eb8f-48df-b06d-c04c05fa0681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:25:59.615935 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05316851-eb8f-48df-b06d-c04c05fa0681 admin admin] PATH_INFO: `/v3/projects/d8b5b40a32894fccb10722c58e023cbe/users/f913685133fa440d8ee08cbaa2fe43a0/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:25:59.615935 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05316851-eb8f-48df-b06d-c04c05fa0681 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8b5b40a32894fccb10722c58e023cbe, user_id=f913685133fa440d8ee08cbaa2fe43a0, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:25:59.615935 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05316851-eb8f-48df-b06d-c04c05fa0681 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:25:59.623401 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1215/2461] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:25:59 2022] PUT /identity/v3/projects/d8b5b40a32894fccb10722c58e023cbe/users/f913685133fa440d8ee08cbaa2fe43a0/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:25:59.639435 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b43abd-bedf-4785-b39b-67f807c91a7c None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:25:59.639730 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b43abd-bedf-4785-b39b-67f807c91a7c None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:25:59.640016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b43abd-bedf-4785-b39b-67f807c91a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:25:59.675021 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-42b43abd-bedf-4785-b39b-67f807c91a7c None None] MFA Rules not processed for user `f913685133fa440d8ee08cbaa2fe43a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:25:59.675021 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-42b43abd-bedf-4785-b39b-67f807c91a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:25:59.717631 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:25:59.772098 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1247/2462] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:25:59 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:25:59.785944 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05316851-eb8f-48df-b06d-c04c05fa0681 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:25:59.787856 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-05316851-eb8f-48df-b06d-c04c05fa0681 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:25:59.863346 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05316851-eb8f-48df-b06d-c04c05fa0681 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:25:59.867040 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b033ee2d-058d-4720-92ad-1f83bade396f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:25:59.867673 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033ee2d-058d-4720-92ad-1f83bade396f service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:25:59.867959 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033ee2d-058d-4720-92ad-1f83bade396f service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:25:59.868235 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033ee2d-058d-4720-92ad-1f83bade396f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:25:59.878699 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b033ee2d-058d-4720-92ad-1f83bade396f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:25:59.880167 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b033ee2d-058d-4720-92ad-1f83bade396f service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:25:59.890294 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1216/2463] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:25:59 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:26:00.002943 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b43abd-bedf-4785-b39b-67f807c91a7c None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:00.004489 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-42b43abd-bedf-4785-b39b-67f807c91a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:00.101153 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b43abd-bedf-4785-b39b-67f807c91a7c None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:00.107409 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba81cf82-349f-4a25-8c96-4ca6ed436238 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:00.108189 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81cf82-349f-4a25-8c96-4ca6ed436238 service nova] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:00.109562 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81cf82-349f-4a25-8c96-4ca6ed436238 service nova] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:00.109763 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81cf82-349f-4a25-8c96-4ca6ed436238 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:00.111664 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-ba81cf82-349f-4a25-8c96-4ca6ed436238 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:00.125144 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba81cf82-349f-4a25-8c96-4ca6ed436238 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:00.126769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba81cf82-349f-4a25-8c96-4ca6ed436238 service nova] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:00.138539 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1248/2464] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:26:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:26:00.902384 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b033ee2d-058d-4720-92ad-1f83bade396f service cinder] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:00.923197 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:26:00.989075 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b033ee2d-058d-4720-92ad-1f83bade396f service cinder] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:00.994431 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d003c3-6094-4e7f-8b39-cd73fcaef207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:00.995103 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d003c3-6094-4e7f-8b39-cd73fcaef207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:00.995388 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d003c3-6094-4e7f-8b39-cd73fcaef207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:00.995723 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d003c3-6094-4e7f-8b39-cd73fcaef207 admin admin] PATH_INFO: `/v3/users/7d9b49e8a6aa46cead8969f4225f6153` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:00.996956 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d003c3-6094-4e7f-8b39-cd73fcaef207 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d9b49e8a6aa46cead8969f4225f6153)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:01.000130 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d003c3-6094-4e7f-8b39-cd73fcaef207 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:01.155280 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-42d003c3-6094-4e7f-8b39-cd73fcaef207 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d9b49e8a6aa46cead8969f4225f6153'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:01.155974 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-42d003c3-6094-4e7f-8b39-cd73fcaef207 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d9b49e8a6aa46cead8969f4225f6153'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:01.156605 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-42d003c3-6094-4e7f-8b39-cd73fcaef207 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d9b49e8a6aa46cead8969f4225f6153'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:01.163875 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-42d003c3-6094-4e7f-8b39-cd73fcaef207 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d9b49e8a6aa46cead8969f4225f6153'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:01.177515 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-42d003c3-6094-4e7f-8b39-cd73fcaef207 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d9b49e8a6aa46cead8969f4225f6153'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:01.257370 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1217/2465] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:26:00 2022] DELETE /identity/v3/users/7d9b49e8a6aa46cead8969f4225f6153 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:26:01.261659 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a44e09-2c78-4547-beb7-b0916fedd1fe None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:01.262181 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a44e09-2c78-4547-beb7-b0916fedd1fe None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:01.262276 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a44e09-2c78-4547-beb7-b0916fedd1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:01.282623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-56a44e09-2c78-4547-beb7-b0916fedd1fe None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:26:01.283333 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-56a44e09-2c78-4547-beb7-b0916fedd1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:01.347079 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1249/2466] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:26:01.356904 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d003c3-6094-4e7f-8b39-cd73fcaef207 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:01.358310 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-42d003c3-6094-4e7f-8b39-cd73fcaef207 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:01.416631 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d003c3-6094-4e7f-8b39-cd73fcaef207 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:01.420340 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7b647-0f27-4a1f-bdab-2a971e200fff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:01.420944 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7b647-0f27-4a1f-bdab-2a971e200fff service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:01.421228 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7b647-0f27-4a1f-bdab-2a971e200fff service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:01.422005 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7b647-0f27-4a1f-bdab-2a971e200fff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:01.431079 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f7b647-0f27-4a1f-bdab-2a971e200fff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:01.432539 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f7b647-0f27-4a1f-bdab-2a971e200fff service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:01.442743 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1218/2467] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:26:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:26:01.829817 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a44e09-2c78-4547-beb7-b0916fedd1fe None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:01.831243 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-56a44e09-2c78-4547-beb7-b0916fedd1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:01.844903 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a44e09-2c78-4547-beb7-b0916fedd1fe None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:01.851114 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ced20-903c-4584-ac92-1634384e4f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:01.851114 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ced20-903c-4584-ac92-1634384e4f7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:01.851447 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ced20-903c-4584-ac92-1634384e4f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:01.851678 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ced20-903c-4584-ac92-1634384e4f7a admin admin] PATH_INFO: `/v3/projects/a6a1abc3ddbc44c782b8f130e6078018` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:01.853572 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538ced20-903c-4584-ac92-1634384e4f7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6a1abc3ddbc44c782b8f130e6078018)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:01.855592 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538ced20-903c-4584-ac92-1634384e4f7a admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:01.936337 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-538ced20-903c-4584-ac92-1634384e4f7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6a1abc3ddbc44c782b8f130e6078018', 'request_id': 'req-538ced20-903c-4584-ac92-1634384e4f7a'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:01.938319 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:26:01.942178 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-538ced20-903c-4584-ac92-1634384e4f7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6a1abc3ddbc44c782b8f130e6078018', 'request_id': 'req-538ced20-903c-4584-ac92-1634384e4f7a'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:01.987507 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-538ced20-903c-4584-ac92-1634384e4f7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6a1abc3ddbc44c782b8f130e6078018', 'request_id': 'req-538ced20-903c-4584-ac92-1634384e4f7a'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:01.999585 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-538ced20-903c-4584-ac92-1634384e4f7a admin admin] The token cache is being invalidate because project a6a1abc3ddbc44c782b8f130e6078018 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:26:02.000212 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-538ced20-903c-4584-ac92-1634384e4f7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:02.001241 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-538ced20-903c-4584-ac92-1634384e4f7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:02.003099 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1250/2468] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:26:01 2022] DELETE /identity/v3/projects/a6a1abc3ddbc44c782b8f130e6078018 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:26:02.007198 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7b647-0f27-4a1f-bdab-2a971e200fff service neutron] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:02.017879 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7b647-0f27-4a1f-bdab-2a971e200fff service neutron] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:02.052334 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:26:02.079188 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b3db2d-621c-4748-87b5-ed35aa185717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:02.079890 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3db2d-621c-4748-87b5-ed35aa185717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:02.080181 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3db2d-621c-4748-87b5-ed35aa185717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:02.080463 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3db2d-621c-4748-87b5-ed35aa185717 admin admin] PATH_INFO: `/v3/users/f913685133fa440d8ee08cbaa2fe43a0` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:02.081613 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b3db2d-621c-4748-87b5-ed35aa185717 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f913685133fa440d8ee08cbaa2fe43a0)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:02.083592 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b3db2d-621c-4748-87b5-ed35aa185717 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:02.230665 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-26b3db2d-621c-4748-87b5-ed35aa185717 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f913685133fa440d8ee08cbaa2fe43a0'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:02.231325 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-26b3db2d-621c-4748-87b5-ed35aa185717 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f913685133fa440d8ee08cbaa2fe43a0'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:02.231962 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-26b3db2d-621c-4748-87b5-ed35aa185717 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f913685133fa440d8ee08cbaa2fe43a0'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:02.239246 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-26b3db2d-621c-4748-87b5-ed35aa185717 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f913685133fa440d8ee08cbaa2fe43a0'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:02.251609 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-26b3db2d-621c-4748-87b5-ed35aa185717 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f913685133fa440d8ee08cbaa2fe43a0'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:02.348449 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1219/2469] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:26:02 2022] DELETE /identity/v3/users/f913685133fa440d8ee08cbaa2fe43a0 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:26:02.797624 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ced20-903c-4584-ac92-1634384e4f7a admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:02.802837 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-538ced20-903c-4584-ac92-1634384e4f7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:02.823356 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ced20-903c-4584-ac92-1634384e4f7a admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:02.888718 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffd091f-c9ef-409b-9e04-328c53ef8d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:02.889145 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd091f-c9ef-409b-9e04-328c53ef8d59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:02.889432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd091f-c9ef-409b-9e04-328c53ef8d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:02.889705 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd091f-c9ef-409b-9e04-328c53ef8d59 admin admin] PATH_INFO: `/v3/projects/d8b5b40a32894fccb10722c58e023cbe` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:02.896036 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffd091f-c9ef-409b-9e04-328c53ef8d59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8b5b40a32894fccb10722c58e023cbe)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:02.898233 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffd091f-c9ef-409b-9e04-328c53ef8d59 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:03.017852 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-8ffd091f-c9ef-409b-9e04-328c53ef8d59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8b5b40a32894fccb10722c58e023cbe', 'request_id': 'req-8ffd091f-c9ef-409b-9e04-328c53ef8d59'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:03.024216 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-8ffd091f-c9ef-409b-9e04-328c53ef8d59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8b5b40a32894fccb10722c58e023cbe', 'request_id': 'req-8ffd091f-c9ef-409b-9e04-328c53ef8d59'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:03.148393 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-8ffd091f-c9ef-409b-9e04-328c53ef8d59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8b5b40a32894fccb10722c58e023cbe', 'request_id': 'req-8ffd091f-c9ef-409b-9e04-328c53ef8d59'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:03.161482 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-8ffd091f-c9ef-409b-9e04-328c53ef8d59 admin admin] The token cache is being invalidate because project d8b5b40a32894fccb10722c58e023cbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:26:03.161776 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-8ffd091f-c9ef-409b-9e04-328c53ef8d59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:03.162463 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-8ffd091f-c9ef-409b-9e04-328c53ef8d59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:03.164071 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1251/2470] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:26:02 2022] DELETE /identity/v3/projects/d8b5b40a32894fccb10722c58e023cbe => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:26:03.168935 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79fc67-9f40-4d48-9b17-d11fb33fa59f None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:03.169429 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79fc67-9f40-4d48-9b17-d11fb33fa59f None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:03.169526 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79fc67-9f40-4d48-9b17-d11fb33fa59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:03.193158 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-9f79fc67-9f40-4d48-9b17-d11fb33fa59f None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:26:03.194391 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-9f79fc67-9f40-4d48-9b17-d11fb33fa59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:03.258353 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1220/2471] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:26:03.266280 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618f8bb-47c2-4800-ade9-ccc74c75804f None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:03.266596 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618f8bb-47c2-4800-ade9-ccc74c75804f None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:03.266878 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618f8bb-47c2-4800-ade9-ccc74c75804f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:03.286974 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-5618f8bb-47c2-4800-ade9-ccc74c75804f None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:26:03.291397 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-5618f8bb-47c2-4800-ade9-ccc74c75804f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:03.296408 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1252/2472] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:26:03.301354 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79fc67-9f40-4d48-9b17-d11fb33fa59f None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:03.311462 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79fc67-9f40-4d48-9b17-d11fb33fa59f None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:03.316239 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea575d75-ca7d-4acc-aaa6-1dbec48e3c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:03.316957 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea575d75-ca7d-4acc-aaa6-1dbec48e3c8e admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:03.317210 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea575d75-ca7d-4acc-aaa6-1dbec48e3c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:03.317388 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea575d75-ca7d-4acc-aaa6-1dbec48e3c8e admin admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:03.318067 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea575d75-ca7d-4acc-aaa6-1dbec48e3c8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:26:03.318383 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea575d75-ca7d-4acc-aaa6-1dbec48e3c8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:03.320015 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea575d75-ca7d-4acc-aaa6-1dbec48e3c8e admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:03.332722 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1221/2473] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:26:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:26:03.337067 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618f8bb-47c2-4800-ade9-ccc74c75804f None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:03.348523 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618f8bb-47c2-4800-ade9-ccc74c75804f None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:03.354223 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a04f5e-e8c6-42ec-aeee-d07ad98bac4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:03.354852 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a04f5e-e8c6-42ec-aeee-d07ad98bac4b admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:03.355139 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a04f5e-e8c6-42ec-aeee-d07ad98bac4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:03.355440 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a04f5e-e8c6-42ec-aeee-d07ad98bac4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:03.356100 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a04f5e-e8c6-42ec-aeee-d07ad98bac4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:03.358104 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a04f5e-e8c6-42ec-aeee-d07ad98bac4b admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:03.498121 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1253/2474] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:26:03 2022] POST /identity/v3/projects => generated 391 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:26:03.502515 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea575d75-ca7d-4acc-aaa6-1dbec48e3c8e admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:03.512991 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea575d75-ca7d-4acc-aaa6-1dbec48e3c8e admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:03.551944 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:26:03.574854 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a58d9e9-5a7c-4330-a2a2-2c0283d9a4e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:03.575493 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58d9e9-5a7c-4330-a2a2-2c0283d9a4e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:03.575778 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58d9e9-5a7c-4330-a2a2-2c0283d9a4e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:03.576056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58d9e9-5a7c-4330-a2a2-2c0283d9a4e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:03.576882 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a58d9e9-5a7c-4330-a2a2-2c0283d9a4e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:03.578762 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a58d9e9-5a7c-4330-a2a2-2c0283d9a4e6 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:03.656719 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1222/2475] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:26:03 2022] POST /identity/v3/users => generated 296 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:26:03.661540 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a04f5e-e8c6-42ec-aeee-d07ad98bac4b admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:03.672463 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a04f5e-e8c6-42ec-aeee-d07ad98bac4b admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:03.679281 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a593d82-2faa-4072-bc5c-8a0d74c9bbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:03.680378 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a593d82-2faa-4072-bc5c-8a0d74c9bbb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:03.680670 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a593d82-2faa-4072-bc5c-8a0d74c9bbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:03.680987 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a593d82-2faa-4072-bc5c-8a0d74c9bbb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:03.682417 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a593d82-2faa-4072-bc5c-8a0d74c9bbb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:03.685023 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a593d82-2faa-4072-bc5c-8a0d74c9bbb5 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:03.703063 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1254/2476] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:26:03.704939 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a58d9e9-5a7c-4330-a2a2-2c0283d9a4e6 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:03.716872 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a58d9e9-5a7c-4330-a2a2-2c0283d9a4e6 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:03.721881 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf98f71-adad-4e1f-838b-d00aae9c84af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:03.722494 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf98f71-adad-4e1f-838b-d00aae9c84af admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:03.722771 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf98f71-adad-4e1f-838b-d00aae9c84af admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:03.723055 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf98f71-adad-4e1f-838b-d00aae9c84af admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:03.723741 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf98f71-adad-4e1f-838b-d00aae9c84af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:03.725719 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf98f71-adad-4e1f-838b-d00aae9c84af admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:03.737528 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1223/2477] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 12:26:03.752041 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a593d82-2faa-4072-bc5c-8a0d74c9bbb5 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:03.753313 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a593d82-2faa-4072-bc5c-8a0d74c9bbb5 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:03.758231 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795fa25-b3ac-4f19-a38f-3dcf0e9c5624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:03.758893 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795fa25-b3ac-4f19-a38f-3dcf0e9c5624 admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:03.759185 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795fa25-b3ac-4f19-a38f-3dcf0e9c5624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:03.759499 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795fa25-b3ac-4f19-a38f-3dcf0e9c5624 admin admin] PATH_INFO: `/v3/projects/a2ac8631af744618b64d867ad407c5fe/users/ec88bbd9e882434cbab7837f3c04e341/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:03.775639 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9795fa25-b3ac-4f19-a38f-3dcf0e9c5624 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2ac8631af744618b64d867ad407c5fe, user_id=ec88bbd9e882434cbab7837f3c04e341, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:03.778293 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9795fa25-b3ac-4f19-a38f-3dcf0e9c5624 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:03.891376 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1255/2478] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:26:03 2022] PUT /identity/v3/projects/a2ac8631af744618b64d867ad407c5fe/users/ec88bbd9e882434cbab7837f3c04e341/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:26:03.904626 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b10a6d-785f-425e-bc54-5257f0e444bb None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:03.904918 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b10a6d-785f-425e-bc54-5257f0e444bb None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:03.905211 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b10a6d-785f-425e-bc54-5257f0e444bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:03.933825 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-a4b10a6d-785f-425e-bc54-5257f0e444bb None None] MFA Rules not processed for user `ec88bbd9e882434cbab7837f3c04e341`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:26:03.935054 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-a4b10a6d-785f-425e-bc54-5257f0e444bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:04.002785 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1224/2479] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:26:03 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:26:04.007107 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795fa25-b3ac-4f19-a38f-3dcf0e9c5624 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:04.016581 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:26:04.070768 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795fa25-b3ac-4f19-a38f-3dcf0e9c5624 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:04.075517 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c501fb29-e4da-47d3-8c63-539b7da1949d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:04.076071 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501fb29-e4da-47d3-8c63-539b7da1949d admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:04.076355 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501fb29-e4da-47d3-8c63-539b7da1949d admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:04.076633 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501fb29-e4da-47d3-8c63-539b7da1949d admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:04.077253 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c501fb29-e4da-47d3-8c63-539b7da1949d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:04.079878 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c501fb29-e4da-47d3-8c63-539b7da1949d admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:04.184535 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1256/2480] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:26:04 2022] POST /identity/v3/projects => generated 393 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:26:04.191613 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b10a6d-785f-425e-bc54-5257f0e444bb None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:04.203139 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b10a6d-785f-425e-bc54-5257f0e444bb None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:04.272522 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1463016-60cb-413d-bdf2-6e42d6962309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:04.273141 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1463016-60cb-413d-bdf2-6e42d6962309 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:04.273426 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1463016-60cb-413d-bdf2-6e42d6962309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:04.273706 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1463016-60cb-413d-bdf2-6e42d6962309 admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:04.274351 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1463016-60cb-413d-bdf2-6e42d6962309 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:04.276314 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1463016-60cb-413d-bdf2-6e42d6962309 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:04.402443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1225/2481] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:26:04 2022] POST /identity/v3/users => generated 303 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:26:04.406864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c501fb29-e4da-47d3-8c63-539b7da1949d admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:04.417944 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c501fb29-e4da-47d3-8c63-539b7da1949d admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:04.423583 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e426a-5d64-4bc0-8d21-5cb9b7b68463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:04.424198 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e426a-5d64-4bc0-8d21-5cb9b7b68463 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:04.424492 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e426a-5d64-4bc0-8d21-5cb9b7b68463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:04.424782 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e426a-5d64-4bc0-8d21-5cb9b7b68463 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:04.425430 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753e426a-5d64-4bc0-8d21-5cb9b7b68463 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:04.427026 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753e426a-5d64-4bc0-8d21-5cb9b7b68463 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:04.442596 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1257/2482] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:26:04.443909 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1463016-60cb-413d-bdf2-6e42d6962309 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:04.453980 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1463016-60cb-413d-bdf2-6e42d6962309 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:04.458792 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6541803-1b10-4ae1-987d-436defc39418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:04.459545 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6541803-1b10-4ae1-987d-436defc39418 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:04.459838 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6541803-1b10-4ae1-987d-436defc39418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:04.460122 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6541803-1b10-4ae1-987d-436defc39418 admin admin] PATH_INFO: `/v3/projects/8da889cce0814d58b1dd35c428bf9608/users/e1353826ba464830887793bbce51d536/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:04.475760 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6541803-1b10-4ae1-987d-436defc39418 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8da889cce0814d58b1dd35c428bf9608, user_id=e1353826ba464830887793bbce51d536, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:04.478495 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6541803-1b10-4ae1-987d-436defc39418 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:04.566129 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1226/2483] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:26:04 2022] PUT /identity/v3/projects/8da889cce0814d58b1dd35c428bf9608/users/e1353826ba464830887793bbce51d536/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:26:04.571037 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e426a-5d64-4bc0-8d21-5cb9b7b68463 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:04.582038 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e426a-5d64-4bc0-8d21-5cb9b7b68463 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:04.643521 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf2b33a-5517-4128-8fe1-1a96ae58f184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:04.644013 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf2b33a-5517-4128-8fe1-1a96ae58f184 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:04.644340 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf2b33a-5517-4128-8fe1-1a96ae58f184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:04.644581 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf2b33a-5517-4128-8fe1-1a96ae58f184 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:04.645290 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf2b33a-5517-4128-8fe1-1a96ae58f184 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:04.646895 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf2b33a-5517-4128-8fe1-1a96ae58f184 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:04.657974 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1258/2484] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 12:26:04.662408 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6541803-1b10-4ae1-987d-436defc39418 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:04.673586 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6541803-1b10-4ae1-987d-436defc39418 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:04.678324 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df76484-616d-43e1-aa7d-c32c97bbd39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:04.678999 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df76484-616d-43e1-aa7d-c32c97bbd39a admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:04.679321 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df76484-616d-43e1-aa7d-c32c97bbd39a admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:04.679630 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df76484-616d-43e1-aa7d-c32c97bbd39a admin admin] PATH_INFO: `/v3/projects/8da889cce0814d58b1dd35c428bf9608/users/e1353826ba464830887793bbce51d536/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:04.681619 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df76484-616d-43e1-aa7d-c32c97bbd39a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8da889cce0814d58b1dd35c428bf9608, user_id=e1353826ba464830887793bbce51d536, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:04.684288 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df76484-616d-43e1-aa7d-c32c97bbd39a admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:04.691400 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:26:04.697036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1227/2485] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:26:04 2022] PUT /identity/v3/projects/8da889cce0814d58b1dd35c428bf9608/users/e1353826ba464830887793bbce51d536/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:26:04.706653 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33fa9d-2776-479c-a620-ab74199cda22 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:04.706950 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33fa9d-2776-479c-a620-ab74199cda22 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:04.707233 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33fa9d-2776-479c-a620-ab74199cda22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:04.737892 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-2e33fa9d-2776-479c-a620-ab74199cda22 None None] MFA Rules not processed for user `e1353826ba464830887793bbce51d536`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:26:04.744082 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-2e33fa9d-2776-479c-a620-ab74199cda22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:04.816976 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1259/2486] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:26:04 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:26:04.825377 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df76484-616d-43e1-aa7d-c32c97bbd39a admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:04.827658 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-2df76484-616d-43e1-aa7d-c32c97bbd39a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:04.888312 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df76484-616d-43e1-aa7d-c32c97bbd39a admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:04.892885 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda00a1-314c-4ec8-81d8-4af30bf4cb9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:04.893491 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda00a1-314c-4ec8-81d8-4af30bf4cb9f service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:04.893824 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda00a1-314c-4ec8-81d8-4af30bf4cb9f service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:04.894048 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda00a1-314c-4ec8-81d8-4af30bf4cb9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:04.903081 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deda00a1-314c-4ec8-81d8-4af30bf4cb9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:04.904562 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deda00a1-314c-4ec8-81d8-4af30bf4cb9f service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:04.914666 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1228/2487] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:26:04 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:26:04.961900 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e33fa9d-2776-479c-a620-ab74199cda22 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:04.963343 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-2e33fa9d-2776-479c-a620-ab74199cda22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:04.971233 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:26:04.976855 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e33fa9d-2776-479c-a620-ab74199cda22 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:04.980490 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81bc18-8a39-4613-a071-3199c59a6d60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:04.981093 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81bc18-8a39-4613-a071-3199c59a6d60 service cinder] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:04.981372 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81bc18-8a39-4613-a071-3199c59a6d60 service cinder] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:04.981645 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81bc18-8a39-4613-a071-3199c59a6d60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:05.045636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d81bc18-8a39-4613-a071-3199c59a6d60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:05.046979 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d81bc18-8a39-4613-a071-3199c59a6d60 service cinder] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:05.057306 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1260/2488] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:26:04 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:26:05.465251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de61b34-4b4d-41d8-8854-caad61a9ed80 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:05.465781 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de61b34-4b4d-41d8-8854-caad61a9ed80 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:05.465902 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de61b34-4b4d-41d8-8854-caad61a9ed80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:05.484949 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-9de61b34-4b4d-41d8-8854-caad61a9ed80 None None] MFA Rules not processed for user `d98949bd624a4330a473ebe5de31586a`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:26:05.486154 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-9de61b34-4b4d-41d8-8854-caad61a9ed80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:05.541472 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1229/2489] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:26:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:26:05.873676 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81bc18-8a39-4613-a071-3199c59a6d60 service cinder] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:05.876388 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-1d81bc18-8a39-4613-a071-3199c59a6d60 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:05.934521 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81bc18-8a39-4613-a071-3199c59a6d60 service cinder] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:05.938205 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1916d4e4-02dc-48dd-827a-ad80796112e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:05.939381 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916d4e4-02dc-48dd-827a-ad80796112e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:05.939687 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916d4e4-02dc-48dd-827a-ad80796112e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:05.939984 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916d4e4-02dc-48dd-827a-ad80796112e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:05.941891 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-1916d4e4-02dc-48dd-827a-ad80796112e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:05.951777 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1916d4e4-02dc-48dd-827a-ad80796112e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:05.953206 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1916d4e4-02dc-48dd-827a-ad80796112e5 service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:05.961574 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1261/2490] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:26:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:26:06.385516 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de61b34-4b4d-41d8-8854-caad61a9ed80 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:06.387030 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-9de61b34-4b4d-41d8-8854-caad61a9ed80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:06.461131 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de61b34-4b4d-41d8-8854-caad61a9ed80 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:06.466189 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc01195-02b0-44b5-bd16-872a61c6e96b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193637acadc14ec4a48ab79566ba3dab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:06.466789 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc01195-02b0-44b5-bd16-872a61c6e96b service placement] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:06.467069 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc01195-02b0-44b5-bd16-872a61c6e96b service placement] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:06.467346 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc01195-02b0-44b5-bd16-872a61c6e96b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:06.469102 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-2bc01195-02b0-44b5-bd16-872a61c6e96b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:06.481258 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc01195-02b0-44b5-bd16-872a61c6e96b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:06.483042 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc01195-02b0-44b5-bd16-872a61c6e96b service placement] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:06.493211 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1230/2491] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:26:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:26:07.498534 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1916d4e4-02dc-48dd-827a-ad80796112e5 service neutron] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:07.500206 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-1916d4e4-02dc-48dd-827a-ad80796112e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:07.600996 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1916d4e4-02dc-48dd-827a-ad80796112e5 service neutron] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:07.610386 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f94a6d-1add-49f2-8907-d5dd8c8e1865 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:07.611592 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f94a6d-1add-49f2-8907-d5dd8c8e1865 service nova] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:07.611879 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f94a6d-1add-49f2-8907-d5dd8c8e1865 service nova] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:07.612154 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f94a6d-1add-49f2-8907-d5dd8c8e1865 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:07.615764 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-e4f94a6d-1add-49f2-8907-d5dd8c8e1865 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:07.665493 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:26:07.687584 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f94a6d-1add-49f2-8907-d5dd8c8e1865 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:07.693460 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f94a6d-1add-49f2-8907-d5dd8c8e1865 service nova] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:07.702116 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1262/2492] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:26:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:26:09.622502 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc01195-02b0-44b5-bd16-872a61c6e96b service placement] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:09.623992 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-2bc01195-02b0-44b5-bd16-872a61c6e96b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:09.638361 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc01195-02b0-44b5-bd16-872a61c6e96b service placement] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:09.642383 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f08bf06-757f-4b36-b5e8-967bc35c9c5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:09.642996 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08bf06-757f-4b36-b5e8-967bc35c9c5c service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:09.643887 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08bf06-757f-4b36-b5e8-967bc35c9c5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:09.644343 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08bf06-757f-4b36-b5e8-967bc35c9c5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:09.646158 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-1f08bf06-757f-4b36-b5e8-967bc35c9c5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:09.660783 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f08bf06-757f-4b36-b5e8-967bc35c9c5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:09.661178 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f08bf06-757f-4b36-b5e8-967bc35c9c5c service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:09.669589 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1231/2493] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:26:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:26:10.136612 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f94a6d-1add-49f2-8907-d5dd8c8e1865 service nova] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:10.202980 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f94a6d-1add-49f2-8907-d5dd8c8e1865 service nova] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:10.210077 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f0fa4b-75dd-4ef6-86fa-0980a603766a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:10.210921 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0fa4b-75dd-4ef6-86fa-0980a603766a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:10.211203 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0fa4b-75dd-4ef6-86fa-0980a603766a admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:10.211569 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0fa4b-75dd-4ef6-86fa-0980a603766a admin admin] PATH_INFO: `/v3/users/ec88bbd9e882434cbab7837f3c04e341` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:10.212818 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f0fa4b-75dd-4ef6-86fa-0980a603766a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec88bbd9e882434cbab7837f3c04e341)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:10.214886 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f0fa4b-75dd-4ef6-86fa-0980a603766a admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:10.456903 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-a2f0fa4b-75dd-4ef6-86fa-0980a603766a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec88bbd9e882434cbab7837f3c04e341'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:10.465766 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-a2f0fa4b-75dd-4ef6-86fa-0980a603766a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec88bbd9e882434cbab7837f3c04e341'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:10.466927 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-a2f0fa4b-75dd-4ef6-86fa-0980a603766a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec88bbd9e882434cbab7837f3c04e341'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:10.484432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-a2f0fa4b-75dd-4ef6-86fa-0980a603766a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec88bbd9e882434cbab7837f3c04e341'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:10.570382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-a2f0fa4b-75dd-4ef6-86fa-0980a603766a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec88bbd9e882434cbab7837f3c04e341'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:10.581825 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1263/2494] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:26:10 2022] DELETE /identity/v3/users/ec88bbd9e882434cbab7837f3c04e341 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:26:10.583008 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d785c-7612-47b0-a25d-4cf44ae79659 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:10.583302 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d785c-7612-47b0-a25d-4cf44ae79659 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:10.583606 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d785c-7612-47b0-a25d-4cf44ae79659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:10.603174 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-f96d785c-7612-47b0-a25d-4cf44ae79659 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:26:10.604408 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-f96d785c-7612-47b0-a25d-4cf44ae79659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:10.624584 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:26:10.666923 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1232/2495] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:26:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:26:10.680027 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f0fa4b-75dd-4ef6-86fa-0980a603766a admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:10.681611 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-a2f0fa4b-75dd-4ef6-86fa-0980a603766a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:10.755436 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f0fa4b-75dd-4ef6-86fa-0980a603766a admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:10.759261 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02fca19-cb0e-4f6c-a321-42373ce76a48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:10.759916 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02fca19-cb0e-4f6c-a321-42373ce76a48 service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:10.760196 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02fca19-cb0e-4f6c-a321-42373ce76a48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:10.760470 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02fca19-cb0e-4f6c-a321-42373ce76a48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:10.771983 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02fca19-cb0e-4f6c-a321-42373ce76a48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:10.773446 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02fca19-cb0e-4f6c-a321-42373ce76a48 service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:10.784853 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1264/2496] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:26:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:26:11.434804 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d785c-7612-47b0-a25d-4cf44ae79659 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:11.436379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-f96d785c-7612-47b0-a25d-4cf44ae79659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:11.453249 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d785c-7612-47b0-a25d-4cf44ae79659 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:11.460187 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a214a4d-0872-47dd-8c72-9a2a932aed70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:11.460862 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a214a4d-0872-47dd-8c72-9a2a932aed70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:11.461281 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a214a4d-0872-47dd-8c72-9a2a932aed70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:11.461487 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a214a4d-0872-47dd-8c72-9a2a932aed70 admin admin] PATH_INFO: `/v3/projects/a2ac8631af744618b64d867ad407c5fe` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:11.462678 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a214a4d-0872-47dd-8c72-9a2a932aed70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2ac8631af744618b64d867ad407c5fe)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:11.464631 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a214a4d-0872-47dd-8c72-9a2a932aed70 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:11.576840 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-9a214a4d-0872-47dd-8c72-9a2a932aed70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2ac8631af744618b64d867ad407c5fe', 'request_id': 'req-9a214a4d-0872-47dd-8c72-9a2a932aed70'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:11.582404 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-9a214a4d-0872-47dd-8c72-9a2a932aed70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2ac8631af744618b64d867ad407c5fe', 'request_id': 'req-9a214a4d-0872-47dd-8c72-9a2a932aed70'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:11.644221 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-9a214a4d-0872-47dd-8c72-9a2a932aed70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2ac8631af744618b64d867ad407c5fe', 'request_id': 'req-9a214a4d-0872-47dd-8c72-9a2a932aed70'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:11.655665 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-9a214a4d-0872-47dd-8c72-9a2a932aed70 admin admin] The token cache is being invalidate because project a2ac8631af744618b64d867ad407c5fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:26:11.655952 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-9a214a4d-0872-47dd-8c72-9a2a932aed70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:11.656579 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-9a214a4d-0872-47dd-8c72-9a2a932aed70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:11.658053 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1233/2497] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:26:11 2022] DELETE /identity/v3/projects/a2ac8631af744618b64d867ad407c5fe => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:26:11.662179 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02fca19-cb0e-4f6c-a321-42373ce76a48 service neutron] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:11.667133 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:26:11.672743 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02fca19-cb0e-4f6c-a321-42373ce76a48 service neutron] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:11.731377 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa0b60-a272-4785-8f17-bc833fc8c75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:11.734143 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa0b60-a272-4785-8f17-bc833fc8c75f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:11.734143 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa0b60-a272-4785-8f17-bc833fc8c75f admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:11.734143 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa0b60-a272-4785-8f17-bc833fc8c75f admin admin] PATH_INFO: `/v3/users/e1353826ba464830887793bbce51d536` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:11.734143 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aa0b60-a272-4785-8f17-bc833fc8c75f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1353826ba464830887793bbce51d536)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:11.736413 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aa0b60-a272-4785-8f17-bc833fc8c75f admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:11.938833 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-23aa0b60-a272-4785-8f17-bc833fc8c75f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1353826ba464830887793bbce51d536'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:11.945770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-23aa0b60-a272-4785-8f17-bc833fc8c75f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1353826ba464830887793bbce51d536'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:11.946476 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-23aa0b60-a272-4785-8f17-bc833fc8c75f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1353826ba464830887793bbce51d536'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:11.958311 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-23aa0b60-a272-4785-8f17-bc833fc8c75f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1353826ba464830887793bbce51d536'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:12.015975 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-23aa0b60-a272-4785-8f17-bc833fc8c75f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1353826ba464830887793bbce51d536'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:12.022128 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1265/2498] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:26:11 2022] DELETE /identity/v3/users/e1353826ba464830887793bbce51d536 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:26:12.543966 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a214a4d-0872-47dd-8c72-9a2a932aed70 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:12.545635 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-9a214a4d-0872-47dd-8c72-9a2a932aed70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:12.560531 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a214a4d-0872-47dd-8c72-9a2a932aed70 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:12.620477 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993c652-49c0-4570-b0f2-161b4fa8d1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:12.621108 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993c652-49c0-4570-b0f2-161b4fa8d1f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:12.621394 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993c652-49c0-4570-b0f2-161b4fa8d1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:12.621672 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993c652-49c0-4570-b0f2-161b4fa8d1f6 admin admin] PATH_INFO: `/v3/projects/8da889cce0814d58b1dd35c428bf9608` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:12.622823 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c993c652-49c0-4570-b0f2-161b4fa8d1f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8da889cce0814d58b1dd35c428bf9608)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:12.624765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c993c652-49c0-4570-b0f2-161b4fa8d1f6 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:12.746363 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-c993c652-49c0-4570-b0f2-161b4fa8d1f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8da889cce0814d58b1dd35c428bf9608', 'request_id': 'req-c993c652-49c0-4570-b0f2-161b4fa8d1f6'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:12.747584 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:26:12.752899 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-c993c652-49c0-4570-b0f2-161b4fa8d1f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8da889cce0814d58b1dd35c428bf9608', 'request_id': 'req-c993c652-49c0-4570-b0f2-161b4fa8d1f6'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:12.861568 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-c993c652-49c0-4570-b0f2-161b4fa8d1f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8da889cce0814d58b1dd35c428bf9608', 'request_id': 'req-c993c652-49c0-4570-b0f2-161b4fa8d1f6'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:12.873463 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-c993c652-49c0-4570-b0f2-161b4fa8d1f6 admin admin] The token cache is being invalidate because project 8da889cce0814d58b1dd35c428bf9608 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:26:12.873758 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-c993c652-49c0-4570-b0f2-161b4fa8d1f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:12.874395 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-c993c652-49c0-4570-b0f2-161b4fa8d1f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:12.875935 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1234/2499] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:26:12 2022] DELETE /identity/v3/projects/8da889cce0814d58b1dd35c428bf9608 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:26:12.881524 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28cf74-4d9c-4ef5-aae7-a36f70fa520d None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:12.882051 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28cf74-4d9c-4ef5-aae7-a36f70fa520d None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:12.882153 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28cf74-4d9c-4ef5-aae7-a36f70fa520d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:12.902134 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-9a28cf74-4d9c-4ef5-aae7-a36f70fa520d None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:26:12.903353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-9a28cf74-4d9c-4ef5-aae7-a36f70fa520d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:12.963143 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1266/2500] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:26:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:26:12.971152 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b11b60-8e6a-4f4e-9740-64a28b06023b None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:12.971498 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b11b60-8e6a-4f4e-9740-64a28b06023b None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:12.971820 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b11b60-8e6a-4f4e-9740-64a28b06023b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:12.991542 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-f6b11b60-8e6a-4f4e-9740-64a28b06023b None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:26:12.992755 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-f6b11b60-8e6a-4f4e-9740-64a28b06023b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:13.000695 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1235/2501] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:26:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:26:13.005423 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a28cf74-4d9c-4ef5-aae7-a36f70fa520d None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:13.014985 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a28cf74-4d9c-4ef5-aae7-a36f70fa520d None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:13.020213 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898dee74-82d1-4f39-9c71-7e490088ec83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:13.020799 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898dee74-82d1-4f39-9c71-7e490088ec83 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:13.021079 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898dee74-82d1-4f39-9c71-7e490088ec83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:13.021412 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898dee74-82d1-4f39-9c71-7e490088ec83 admin admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:13.022049 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898dee74-82d1-4f39-9c71-7e490088ec83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:26:13.022353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898dee74-82d1-4f39-9c71-7e490088ec83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:13.023994 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898dee74-82d1-4f39-9c71-7e490088ec83 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:13.035240 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1267/2502] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:26:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:26:13.039696 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b11b60-8e6a-4f4e-9740-64a28b06023b None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:13.049054 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b11b60-8e6a-4f4e-9740-64a28b06023b None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:13.054293 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e26ab-a1af-4e00-8b53-83e0b2147021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:13.054892 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e26ab-a1af-4e00-8b53-83e0b2147021 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:13.055175 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e26ab-a1af-4e00-8b53-83e0b2147021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:13.055513 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e26ab-a1af-4e00-8b53-83e0b2147021 admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:13.056156 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6e26ab-a1af-4e00-8b53-83e0b2147021 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:13.058218 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6e26ab-a1af-4e00-8b53-83e0b2147021 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:13.143095 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1236/2503] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:26:13 2022] POST /identity/v3/projects => generated 407 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:26:13.147443 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898dee74-82d1-4f39-9c71-7e490088ec83 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:13.157894 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898dee74-82d1-4f39-9c71-7e490088ec83 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:13.162661 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:26:13.215199 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a227a-ddee-496c-960a-665a967fc34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:13.215850 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a227a-ddee-496c-960a-665a967fc34d admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:13.216133 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a227a-ddee-496c-960a-665a967fc34d admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:13.216414 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a227a-ddee-496c-960a-665a967fc34d admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:13.217052 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876a227a-ddee-496c-960a-665a967fc34d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:13.219061 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876a227a-ddee-496c-960a-665a967fc34d admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:13.297230 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1268/2504] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:26:13 2022] POST /identity/v3/users => generated 304 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:26:13.301921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e26ab-a1af-4e00-8b53-83e0b2147021 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:13.312963 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e26ab-a1af-4e00-8b53-83e0b2147021 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:13.318170 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1651bc-562f-4dd3-bbb2-2de6e6a686ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:13.318774 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1651bc-562f-4dd3-bbb2-2de6e6a686ea admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:13.319054 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1651bc-562f-4dd3-bbb2-2de6e6a686ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:13.319331 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1651bc-562f-4dd3-bbb2-2de6e6a686ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:13.320018 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1651bc-562f-4dd3-bbb2-2de6e6a686ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:13.321596 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1651bc-562f-4dd3-bbb2-2de6e6a686ea admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:13.335226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1237/2505] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:26:13.336455 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a227a-ddee-496c-960a-665a967fc34d admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:13.347702 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a227a-ddee-496c-960a-665a967fc34d admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:13.353031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0646fa3-0056-4683-93d4-87b49fb67588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:13.353031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0646fa3-0056-4683-93d4-87b49fb67588 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:13.353344 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0646fa3-0056-4683-93d4-87b49fb67588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:13.353593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0646fa3-0056-4683-93d4-87b49fb67588 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:13.354245 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0646fa3-0056-4683-93d4-87b49fb67588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:13.355887 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0646fa3-0056-4683-93d4-87b49fb67588 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:13.368516 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1269/2506] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:26:13.374489 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1651bc-562f-4dd3-bbb2-2de6e6a686ea admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:13.385638 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1651bc-562f-4dd3-bbb2-2de6e6a686ea admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:13.390458 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f61f3-5825-4d9a-abb9-f6db9160d929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:13.391214 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f61f3-5825-4d9a-abb9-f6db9160d929 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:13.391603 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f61f3-5825-4d9a-abb9-f6db9160d929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:13.391952 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f61f3-5825-4d9a-abb9-f6db9160d929 admin admin] PATH_INFO: `/v3/projects/5310bfd767d747a88b1aaefe039856c1/users/0d115f35b20e4a7dbe979e7adc6d1eb2/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:13.411532 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f61f3-5825-4d9a-abb9-f6db9160d929 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5310bfd767d747a88b1aaefe039856c1, user_id=0d115f35b20e4a7dbe979e7adc6d1eb2, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:13.414156 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f61f3-5825-4d9a-abb9-f6db9160d929 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:13.507798 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1238/2507] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:26:13 2022] PUT /identity/v3/projects/5310bfd767d747a88b1aaefe039856c1/users/0d115f35b20e4a7dbe979e7adc6d1eb2/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:26:13.519381 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345ecd0-4d83-44b1-b654-1aadc8b8d6d0 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:13.519534 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345ecd0-4d83-44b1-b654-1aadc8b8d6d0 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:13.519881 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345ecd0-4d83-44b1-b654-1aadc8b8d6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:13.553999 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-7345ecd0-4d83-44b1-b654-1aadc8b8d6d0 None None] MFA Rules not processed for user `0d115f35b20e4a7dbe979e7adc6d1eb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:26:13.555236 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-7345ecd0-4d83-44b1-b654-1aadc8b8d6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:13.627229 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1270/2508] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:26:13 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:26:13.631721 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f61f3-5825-4d9a-abb9-f6db9160d929 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:13.704783 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f61f3-5825-4d9a-abb9-f6db9160d929 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:13.709519 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b705ea-a434-47b7-b766-6f440b7852ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:13.710126 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b705ea-a434-47b7-b766-6f440b7852ff admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:13.710407 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b705ea-a434-47b7-b766-6f440b7852ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:13.710680 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b705ea-a434-47b7-b766-6f440b7852ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:13.711304 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b705ea-a434-47b7-b766-6f440b7852ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:13.713237 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b705ea-a434-47b7-b766-6f440b7852ff admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:13.828632 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1239/2509] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:26:13 2022] POST /identity/v3/projects => generated 409 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:26:13.837695 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7345ecd0-4d83-44b1-b654-1aadc8b8d6d0 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:13.850502 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7345ecd0-4d83-44b1-b654-1aadc8b8d6d0 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:13.910084 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4d10c8-be0b-414b-9239-c420ac1a0b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:13.910865 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d10c8-be0b-414b-9239-c420ac1a0b05 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:13.910967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d10c8-be0b-414b-9239-c420ac1a0b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:13.911233 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d10c8-be0b-414b-9239-c420ac1a0b05 admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:13.911906 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4d10c8-be0b-414b-9239-c420ac1a0b05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:13.913832 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4d10c8-be0b-414b-9239-c420ac1a0b05 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:14.032353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1271/2510] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:26:13 2022] POST /identity/v3/users => generated 311 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:26:14.037046 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b705ea-a434-47b7-b766-6f440b7852ff admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:14.050560 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b705ea-a434-47b7-b766-6f440b7852ff admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:14.055259 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b093fe17-d9e6-49f9-b932-8f4d784b93e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:14.055884 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093fe17-d9e6-49f9-b932-8f4d784b93e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:14.056173 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093fe17-d9e6-49f9-b932-8f4d784b93e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:14.056451 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093fe17-d9e6-49f9-b932-8f4d784b93e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:14.057099 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b093fe17-d9e6-49f9-b932-8f4d784b93e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:14.058754 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b093fe17-d9e6-49f9-b932-8f4d784b93e9 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:14.069846 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1240/2511] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 12:26:14.074187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4d10c8-be0b-414b-9239-c420ac1a0b05 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:14.085000 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4d10c8-be0b-414b-9239-c420ac1a0b05 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:14.089790 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647fe18f-6d86-4b5a-ada7-093c614d0a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:14.090444 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647fe18f-6d86-4b5a-ada7-093c614d0a39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:14.090733 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647fe18f-6d86-4b5a-ada7-093c614d0a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:14.091017 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647fe18f-6d86-4b5a-ada7-093c614d0a39 admin admin] PATH_INFO: `/v3/projects/7b0f48a481dd451ea71f02a901ff15be/users/b026d6d6fd13439bbae619f1ecc7c3f3/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:14.093309 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:26:14.107093 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647fe18f-6d86-4b5a-ada7-093c614d0a39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b0f48a481dd451ea71f02a901ff15be, user_id=b026d6d6fd13439bbae619f1ecc7c3f3, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:14.110076 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647fe18f-6d86-4b5a-ada7-093c614d0a39 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:14.209745 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1272/2512] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:26:14 2022] PUT /identity/v3/projects/7b0f48a481dd451ea71f02a901ff15be/users/b026d6d6fd13439bbae619f1ecc7c3f3/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:26:14.214205 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b093fe17-d9e6-49f9-b932-8f4d784b93e9 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:14.224409 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b093fe17-d9e6-49f9-b932-8f4d784b93e9 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:14.269308 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:26:14.280981 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd2805-7cae-4aea-b5c6-62f1376a9f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:14.281591 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd2805-7cae-4aea-b5c6-62f1376a9f86 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:14.282280 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd2805-7cae-4aea-b5c6-62f1376a9f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:14.282280 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd2805-7cae-4aea-b5c6-62f1376a9f86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:14.282810 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd2805-7cae-4aea-b5c6-62f1376a9f86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:14.284396 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd2805-7cae-4aea-b5c6-62f1376a9f86 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:14.293971 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1241/2513] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:26:14.298280 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647fe18f-6d86-4b5a-ada7-093c614d0a39 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:14.308868 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647fe18f-6d86-4b5a-ada7-093c614d0a39 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:14.313462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec9ffef-56e5-4eb7-9089-804e1867914a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:14.314102 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9ffef-56e5-4eb7-9089-804e1867914a admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:14.314382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9ffef-56e5-4eb7-9089-804e1867914a admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:14.314656 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9ffef-56e5-4eb7-9089-804e1867914a admin admin] PATH_INFO: `/v3/projects/7b0f48a481dd451ea71f02a901ff15be/users/b026d6d6fd13439bbae619f1ecc7c3f3/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:14.317147 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec9ffef-56e5-4eb7-9089-804e1867914a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b0f48a481dd451ea71f02a901ff15be, user_id=b026d6d6fd13439bbae619f1ecc7c3f3, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:14.319802 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec9ffef-56e5-4eb7-9089-804e1867914a admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:14.330082 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1273/2514] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:26:14 2022] PUT /identity/v3/projects/7b0f48a481dd451ea71f02a901ff15be/users/b026d6d6fd13439bbae619f1ecc7c3f3/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:26:14.341280 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf09c0-f5af-4af8-9593-69492b9e3af5 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:14.341961 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf09c0-f5af-4af8-9593-69492b9e3af5 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:14.341961 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf09c0-f5af-4af8-9593-69492b9e3af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:14.372332 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-6faf09c0-f5af-4af8-9593-69492b9e3af5 None None] MFA Rules not processed for user `b026d6d6fd13439bbae619f1ecc7c3f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:26:14.373402 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-6faf09c0-f5af-4af8-9593-69492b9e3af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:14.442647 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1242/2515] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:26:14 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:26:14.453732 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec9ffef-56e5-4eb7-9089-804e1867914a admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:14.455482 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-fec9ffef-56e5-4eb7-9089-804e1867914a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:14.514391 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec9ffef-56e5-4eb7-9089-804e1867914a admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:14.518149 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a2f63-f349-44d9-b7e8-f9e83f55b9bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:14.519326 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a2f63-f349-44d9-b7e8-f9e83f55b9bf service cinder] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:14.519636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a2f63-f349-44d9-b7e8-f9e83f55b9bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:14.519911 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a2f63-f349-44d9-b7e8-f9e83f55b9bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:14.529410 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a2f63-f349-44d9-b7e8-f9e83f55b9bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:14.530857 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a2f63-f349-44d9-b7e8-f9e83f55b9bf service cinder] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:14.540736 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1274/2516] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:26:14 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:26:14.883606 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf09c0-f5af-4af8-9593-69492b9e3af5 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:14.893187 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf09c0-f5af-4af8-9593-69492b9e3af5 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:14.896725 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4125915d-3b0f-4ed6-8154-877a926e5180 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:14.897325 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125915d-3b0f-4ed6-8154-877a926e5180 service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:14.897606 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125915d-3b0f-4ed6-8154-877a926e5180 service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:14.897881 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125915d-3b0f-4ed6-8154-877a926e5180 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:14.954505 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4125915d-3b0f-4ed6-8154-877a926e5180 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:14.956009 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4125915d-3b0f-4ed6-8154-877a926e5180 service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:14.965037 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1243/2517] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:26:14 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:26:15.477897 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a2f63-f349-44d9-b7e8-f9e83f55b9bf service cinder] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:15.479445 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-d03a2f63-f349-44d9-b7e8-f9e83f55b9bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:15.554288 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a2f63-f349-44d9-b7e8-f9e83f55b9bf service cinder] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:15.560693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a723d3-92a1-4df7-a8e3-7f59550eecf5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:15.561343 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a723d3-92a1-4df7-a8e3-7f59550eecf5 service nova] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:15.561627 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a723d3-92a1-4df7-a8e3-7f59550eecf5 service nova] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:15.561906 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a723d3-92a1-4df7-a8e3-7f59550eecf5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:15.563731 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-d3a723d3-92a1-4df7-a8e3-7f59550eecf5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:15.577722 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a723d3-92a1-4df7-a8e3-7f59550eecf5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:15.579369 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a723d3-92a1-4df7-a8e3-7f59550eecf5 service nova] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:15.591043 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1275/2518] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:26:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:26:19.427530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4125915d-3b0f-4ed6-8154-877a926e5180 service cinder] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:19.429047 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-4125915d-3b0f-4ed6-8154-877a926e5180 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:19.445203 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4125915d-3b0f-4ed6-8154-877a926e5180 service cinder] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:19.451057 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d5589-4ac3-4491-9597-b2d3ee7af448 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:19.451700 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d5589-4ac3-4491-9597-b2d3ee7af448 service nova] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:19.451984 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d5589-4ac3-4491-9597-b2d3ee7af448 service nova] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:19.452257 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d5589-4ac3-4491-9597-b2d3ee7af448 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:19.453992 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-a48d5589-4ac3-4491-9597-b2d3ee7af448 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:19.466430 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d5589-4ac3-4491-9597-b2d3ee7af448 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:19.468060 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d5589-4ac3-4491-9597-b2d3ee7af448 service nova] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:19.479175 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1244/2519] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:26:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:26:24.882308 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a723d3-92a1-4df7-a8e3-7f59550eecf5 service nova] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:24.939363 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:26:24.943623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a723d3-92a1-4df7-a8e3-7f59550eecf5 service nova] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:24.948688 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fdff73-2dba-4aa3-a124-bb71e97b802b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:24.949352 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdff73-2dba-4aa3-a124-bb71e97b802b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:24.958534 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdff73-2dba-4aa3-a124-bb71e97b802b admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:24.958534 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdff73-2dba-4aa3-a124-bb71e97b802b admin admin] PATH_INFO: `/v3/users/0d115f35b20e4a7dbe979e7adc6d1eb2` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:24.958534 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fdff73-2dba-4aa3-a124-bb71e97b802b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d115f35b20e4a7dbe979e7adc6d1eb2)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:24.958534 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fdff73-2dba-4aa3-a124-bb71e97b802b admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:25.156055 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-15fdff73-2dba-4aa3-a124-bb71e97b802b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d115f35b20e4a7dbe979e7adc6d1eb2'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:25.163560 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-15fdff73-2dba-4aa3-a124-bb71e97b802b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d115f35b20e4a7dbe979e7adc6d1eb2'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:25.164248 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-15fdff73-2dba-4aa3-a124-bb71e97b802b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d115f35b20e4a7dbe979e7adc6d1eb2'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:25.177475 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-15fdff73-2dba-4aa3-a124-bb71e97b802b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d115f35b20e4a7dbe979e7adc6d1eb2'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:25.254233 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-15fdff73-2dba-4aa3-a124-bb71e97b802b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d115f35b20e4a7dbe979e7adc6d1eb2'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:25.260725 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1276/2520] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:26:24 2022] DELETE /identity/v3/users/0d115f35b20e4a7dbe979e7adc6d1eb2 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:26:25.265221 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02df16-d624-472f-b634-fde72a606ec1 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:25.267184 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02df16-d624-472f-b634-fde72a606ec1 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:25.267184 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02df16-d624-472f-b634-fde72a606ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:25.286072 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-ab02df16-d624-472f-b634-fde72a606ec1 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:26:25.287302 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-ab02df16-d624-472f-b634-fde72a606ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:25.334698 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:26:25.345695 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1245/2521] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:26:25.356038 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fdff73-2dba-4aa3-a124-bb71e97b802b admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:25.357774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-15fdff73-2dba-4aa3-a124-bb71e97b802b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:25.416486 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fdff73-2dba-4aa3-a124-bb71e97b802b admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:25.420056 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f58845-327b-4424-ae10-62c068bf4827 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:25.420663 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f58845-327b-4424-ae10-62c068bf4827 service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:25.420950 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f58845-327b-4424-ae10-62c068bf4827 service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:25.421231 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f58845-327b-4424-ae10-62c068bf4827 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:25.432446 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f58845-327b-4424-ae10-62c068bf4827 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:25.433880 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f58845-327b-4424-ae10-62c068bf4827 service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:25.443686 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1277/2522] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:26:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:26:25.891728 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab02df16-d624-472f-b634-fde72a606ec1 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:25.893699 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-ab02df16-d624-472f-b634-fde72a606ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:25.905686 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab02df16-d624-472f-b634-fde72a606ec1 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:25.910999 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6627681-5570-4293-a372-db66d39ea4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:25.911729 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6627681-5570-4293-a372-db66d39ea4d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:25.912219 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6627681-5570-4293-a372-db66d39ea4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:25.912602 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6627681-5570-4293-a372-db66d39ea4d6 admin admin] PATH_INFO: `/v3/projects/5310bfd767d747a88b1aaefe039856c1` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:25.913749 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6627681-5570-4293-a372-db66d39ea4d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5310bfd767d747a88b1aaefe039856c1)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:25.915835 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6627681-5570-4293-a372-db66d39ea4d6 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:26.020856 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-b6627681-5570-4293-a372-db66d39ea4d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5310bfd767d747a88b1aaefe039856c1', 'request_id': 'req-b6627681-5570-4293-a372-db66d39ea4d6'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:26.026597 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-b6627681-5570-4293-a372-db66d39ea4d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5310bfd767d747a88b1aaefe039856c1', 'request_id': 'req-b6627681-5570-4293-a372-db66d39ea4d6'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:26.082546 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-b6627681-5570-4293-a372-db66d39ea4d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5310bfd767d747a88b1aaefe039856c1', 'request_id': 'req-b6627681-5570-4293-a372-db66d39ea4d6'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:26.093925 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-b6627681-5570-4293-a372-db66d39ea4d6 admin admin] The token cache is being invalidate because project 5310bfd767d747a88b1aaefe039856c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:26:26.094218 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-b6627681-5570-4293-a372-db66d39ea4d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:26.094855 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-b6627681-5570-4293-a372-db66d39ea4d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:26.097007 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1246/2523] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:26:25 2022] DELETE /identity/v3/projects/5310bfd767d747a88b1aaefe039856c1 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:26:26.101078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f58845-327b-4424-ae10-62c068bf4827 service neutron] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:26.110947 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f58845-327b-4424-ae10-62c068bf4827 service neutron] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:26.166284 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a7102-4708-4225-8baa-58fccb4bcac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:26.166916 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a7102-4708-4225-8baa-58fccb4bcac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:26.167252 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a7102-4708-4225-8baa-58fccb4bcac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:26.167513 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a7102-4708-4225-8baa-58fccb4bcac5 admin admin] PATH_INFO: `/v3/users/b026d6d6fd13439bbae619f1ecc7c3f3` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:26.168674 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a7102-4708-4225-8baa-58fccb4bcac5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b026d6d6fd13439bbae619f1ecc7c3f3)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:26.170601 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a7102-4708-4225-8baa-58fccb4bcac5 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:26.192189 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:26:26.355547 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-397a7102-4708-4225-8baa-58fccb4bcac5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b026d6d6fd13439bbae619f1ecc7c3f3'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:26.362779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-397a7102-4708-4225-8baa-58fccb4bcac5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b026d6d6fd13439bbae619f1ecc7c3f3'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:26.363482 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-397a7102-4708-4225-8baa-58fccb4bcac5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b026d6d6fd13439bbae619f1ecc7c3f3'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:26.377976 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-397a7102-4708-4225-8baa-58fccb4bcac5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b026d6d6fd13439bbae619f1ecc7c3f3'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:26.430743 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-397a7102-4708-4225-8baa-58fccb4bcac5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b026d6d6fd13439bbae619f1ecc7c3f3'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:26.437012 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1278/2524] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:26:26 2022] DELETE /identity/v3/users/b026d6d6fd13439bbae619f1ecc7c3f3 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:26:26.821668 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6627681-5570-4293-a372-db66d39ea4d6 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:26.823212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-b6627681-5570-4293-a372-db66d39ea4d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:26.837013 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6627681-5570-4293-a372-db66d39ea4d6 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:26.896566 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44562050-b152-485b-9f25-559aaf812511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:26.897199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44562050-b152-485b-9f25-559aaf812511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:26.897484 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44562050-b152-485b-9f25-559aaf812511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:26.897765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44562050-b152-485b-9f25-559aaf812511 admin admin] PATH_INFO: `/v3/projects/7b0f48a481dd451ea71f02a901ff15be` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:26.898970 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44562050-b152-485b-9f25-559aaf812511 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b0f48a481dd451ea71f02a901ff15be)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:26.900922 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44562050-b152-485b-9f25-559aaf812511 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:27.033879 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-44562050-b152-485b-9f25-559aaf812511 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b0f48a481dd451ea71f02a901ff15be', 'request_id': 'req-44562050-b152-485b-9f25-559aaf812511'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:27.039304 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-44562050-b152-485b-9f25-559aaf812511 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b0f48a481dd451ea71f02a901ff15be', 'request_id': 'req-44562050-b152-485b-9f25-559aaf812511'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:27.106896 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-44562050-b152-485b-9f25-559aaf812511 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b0f48a481dd451ea71f02a901ff15be', 'request_id': 'req-44562050-b152-485b-9f25-559aaf812511'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:27.119276 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-44562050-b152-485b-9f25-559aaf812511 admin admin] The token cache is being invalidate because project 7b0f48a481dd451ea71f02a901ff15be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:26:27.119702 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-44562050-b152-485b-9f25-559aaf812511 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:27.120346 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-44562050-b152-485b-9f25-559aaf812511 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:27.121836 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1247/2525] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:26:26 2022] DELETE /identity/v3/projects/7b0f48a481dd451ea71f02a901ff15be => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:26:27.126726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68194d5e-7f6a-47ce-906f-e77fcc0d12a9 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:27.127185 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68194d5e-7f6a-47ce-906f-e77fcc0d12a9 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:27.127278 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68194d5e-7f6a-47ce-906f-e77fcc0d12a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:27.146550 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-68194d5e-7f6a-47ce-906f-e77fcc0d12a9 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:26:27.147809 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-68194d5e-7f6a-47ce-906f-e77fcc0d12a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:27.207118 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1279/2526] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:26:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:26:27.215267 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90dbc8-95ea-4dfd-8e93-3ac69ee27986 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:27.215587 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90dbc8-95ea-4dfd-8e93-3ac69ee27986 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:27.215870 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90dbc8-95ea-4dfd-8e93-3ac69ee27986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:27.238778 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-3d90dbc8-95ea-4dfd-8e93-3ac69ee27986 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:26:27.240181 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-3d90dbc8-95ea-4dfd-8e93-3ac69ee27986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:27.248281 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1248/2527] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:26:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:26:27.252492 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68194d5e-7f6a-47ce-906f-e77fcc0d12a9 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:27.262763 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68194d5e-7f6a-47ce-906f-e77fcc0d12a9 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:27.268207 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9898f2b-caa7-42eb-92e4-d3961294750e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:27.268837 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9898f2b-caa7-42eb-92e4-d3961294750e admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:27.269240 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9898f2b-caa7-42eb-92e4-d3961294750e admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:27.269519 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9898f2b-caa7-42eb-92e4-d3961294750e admin admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:27.270169 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9898f2b-caa7-42eb-92e4-d3961294750e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:26:27.270478 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9898f2b-caa7-42eb-92e4-d3961294750e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:27.272108 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9898f2b-caa7-42eb-92e4-d3961294750e admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:27.284672 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1280/2528] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:26:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:26:27.289130 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d90dbc8-95ea-4dfd-8e93-3ac69ee27986 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:27.293573 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:26:27.298778 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d90dbc8-95ea-4dfd-8e93-3ac69ee27986 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:27.303867 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5050cf6-7a02-422c-aa2f-64c0f084acc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:27.304467 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5050cf6-7a02-422c-aa2f-64c0f084acc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:27.304752 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5050cf6-7a02-422c-aa2f-64c0f084acc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:27.305028 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5050cf6-7a02-422c-aa2f-64c0f084acc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:27.305655 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5050cf6-7a02-422c-aa2f-64c0f084acc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:27.307612 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5050cf6-7a02-422c-aa2f-64c0f084acc1 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:27.418089 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1249/2529] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:26:27 2022] POST /identity/v3/projects => generated 375 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:26:27.422560 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9898f2b-caa7-42eb-92e4-d3961294750e admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:27.433017 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9898f2b-caa7-42eb-92e4-d3961294750e admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:27.490369 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a009f2-3798-48be-a9f1-965af332b5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:27.490892 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a009f2-3798-48be-a9f1-965af332b5e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:27.491179 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a009f2-3798-48be-a9f1-965af332b5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:27.491476 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a009f2-3798-48be-a9f1-965af332b5e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:27.492110 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a009f2-3798-48be-a9f1-965af332b5e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:27.494044 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a009f2-3798-48be-a9f1-965af332b5e0 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:27.564409 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1281/2530] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:26:27 2022] POST /identity/v3/users => generated 288 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:26:27.568578 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5050cf6-7a02-422c-aa2f-64c0f084acc1 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:27.578903 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5050cf6-7a02-422c-aa2f-64c0f084acc1 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:27.584176 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302362c9-8626-4750-86a1-58c81e1b4b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:27.584777 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302362c9-8626-4750-86a1-58c81e1b4b4c admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:27.585062 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302362c9-8626-4750-86a1-58c81e1b4b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:27.585338 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302362c9-8626-4750-86a1-58c81e1b4b4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:27.586015 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302362c9-8626-4750-86a1-58c81e1b4b4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:27.587639 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302362c9-8626-4750-86a1-58c81e1b4b4c admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:27.598052 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1250/2531] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:26:27.602044 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a009f2-3798-48be-a9f1-965af332b5e0 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:27.612431 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a009f2-3798-48be-a9f1-965af332b5e0 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:27.617228 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cbd748-7638-4267-9711-b172bf50a515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:27.617816 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cbd748-7638-4267-9711-b172bf50a515 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:27.618095 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cbd748-7638-4267-9711-b172bf50a515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:27.618370 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cbd748-7638-4267-9711-b172bf50a515 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:27.619011 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cbd748-7638-4267-9711-b172bf50a515 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:27.620662 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cbd748-7638-4267-9711-b172bf50a515 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:27.632059 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1282/2532] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:26:27.636262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302362c9-8626-4750-86a1-58c81e1b4b4c admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:27.646813 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302362c9-8626-4750-86a1-58c81e1b4b4c admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:27.651337 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da9a969-fd6c-463f-a962-47657a3c3079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:27.652042 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9a969-fd6c-463f-a962-47657a3c3079 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:27.652324 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9a969-fd6c-463f-a962-47657a3c3079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:27.652597 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9a969-fd6c-463f-a962-47657a3c3079 admin admin] PATH_INFO: `/v3/projects/74f076c1966040abb26fc3641f4e164f/users/bdef7549a5d544bdb86ed2bfc933d5aa/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:27.666921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da9a969-fd6c-463f-a962-47657a3c3079 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74f076c1966040abb26fc3641f4e164f, user_id=bdef7549a5d544bdb86ed2bfc933d5aa, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:27.669572 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da9a969-fd6c-463f-a962-47657a3c3079 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:27.741873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1251/2533] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:26:27 2022] PUT /identity/v3/projects/74f076c1966040abb26fc3641f4e164f/users/bdef7549a5d544bdb86ed2bfc933d5aa/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:26:27.753827 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2516d-6820-48a7-b130-587cf5dac41a None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:27.754120 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2516d-6820-48a7-b130-587cf5dac41a None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:27.754389 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2516d-6820-48a7-b130-587cf5dac41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:27.762803 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:26:27.785311 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-44f2516d-6820-48a7-b130-587cf5dac41a None None] MFA Rules not processed for user `bdef7549a5d544bdb86ed2bfc933d5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:26:27.786546 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-44f2516d-6820-48a7-b130-587cf5dac41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:27.851842 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1283/2534] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:26:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:26:27.863302 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da9a969-fd6c-463f-a962-47657a3c3079 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:27.865134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-2da9a969-fd6c-463f-a962-47657a3c3079 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:27.924179 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da9a969-fd6c-463f-a962-47657a3c3079 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:27.927893 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d16e5d-da63-4be7-b1a7-085c8b73e20b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd020063702b0481e9f4111680766f7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:27.928505 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d16e5d-da63-4be7-b1a7-085c8b73e20b service glance] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:27.928788 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d16e5d-da63-4be7-b1a7-085c8b73e20b service glance] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:27.929065 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d16e5d-da63-4be7-b1a7-085c8b73e20b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:27.937718 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d16e5d-da63-4be7-b1a7-085c8b73e20b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:27.939155 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d16e5d-da63-4be7-b1a7-085c8b73e20b service glance] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:27.949111 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1252/2535] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:26:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:26:37.906640 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=30 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:26:37.944580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=49,keystone:INSERT=2 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:26:45.411723 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2516d-6820-48a7-b130-587cf5dac41a None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:45.413224 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-44f2516d-6820-48a7-b130-587cf5dac41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:45.471122 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2516d-6820-48a7-b130-587cf5dac41a None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:45.475005 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c94e848-afce-425d-a925-ceda0dd2978b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:45.475650 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94e848-afce-425d-a925-ceda0dd2978b service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:45.475935 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94e848-afce-425d-a925-ceda0dd2978b service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:45.476254 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94e848-afce-425d-a925-ceda0dd2978b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:45.477996 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-7c94e848-afce-425d-a925-ceda0dd2978b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:45.541357 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c94e848-afce-425d-a925-ceda0dd2978b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:45.542920 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c94e848-afce-425d-a925-ceda0dd2978b service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:45.553396 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1284/2536] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:26:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:26:46.106744 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d16e5d-da63-4be7-b1a7-085c8b73e20b service glance] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:46.117656 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d16e5d-da63-4be7-b1a7-085c8b73e20b service glance] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:46.122794 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262f2618-a0a3-4cc6-92ec-8955664f21df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:46.123506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262f2618-a0a3-4cc6-92ec-8955664f21df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:46.123970 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262f2618-a0a3-4cc6-92ec-8955664f21df admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:46.124252 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262f2618-a0a3-4cc6-92ec-8955664f21df admin admin] PATH_INFO: `/v3/users/bdef7549a5d544bdb86ed2bfc933d5aa` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:46.125396 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262f2618-a0a3-4cc6-92ec-8955664f21df admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdef7549a5d544bdb86ed2bfc933d5aa)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:46.127299 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262f2618-a0a3-4cc6-92ec-8955664f21df admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:46.341105 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-262f2618-a0a3-4cc6-92ec-8955664f21df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdef7549a5d544bdb86ed2bfc933d5aa'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:46.341717 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-262f2618-a0a3-4cc6-92ec-8955664f21df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdef7549a5d544bdb86ed2bfc933d5aa'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:46.342425 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-262f2618-a0a3-4cc6-92ec-8955664f21df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdef7549a5d544bdb86ed2bfc933d5aa'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:46.349971 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-262f2618-a0a3-4cc6-92ec-8955664f21df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdef7549a5d544bdb86ed2bfc933d5aa'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:46.363514 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-262f2618-a0a3-4cc6-92ec-8955664f21df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdef7549a5d544bdb86ed2bfc933d5aa'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:46.481525 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1253/2537] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:26:46 2022] DELETE /identity/v3/users/bdef7549a5d544bdb86ed2bfc933d5aa => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:26:46.483007 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48132c1-b8c5-4bc9-a60b-19a04e798082 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:46.483722 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48132c1-b8c5-4bc9-a60b-19a04e798082 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:46.484277 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48132c1-b8c5-4bc9-a60b-19a04e798082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:46.506785 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-b48132c1-b8c5-4bc9-a60b-19a04e798082 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:26:46.508115 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-b48132c1-b8c5-4bc9-a60b-19a04e798082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:46.579177 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1285/2538] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:26:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:26:46.589335 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262f2618-a0a3-4cc6-92ec-8955664f21df admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:46.591506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-262f2618-a0a3-4cc6-92ec-8955664f21df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:46.678600 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262f2618-a0a3-4cc6-92ec-8955664f21df admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:46.694306 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513acbf2-402c-43b3-bd55-a1522e92df56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:46.694306 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513acbf2-402c-43b3-bd55-a1522e92df56 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:46.694306 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513acbf2-402c-43b3-bd55-a1522e92df56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:46.694306 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513acbf2-402c-43b3-bd55-a1522e92df56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:46.697624 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513acbf2-402c-43b3-bd55-a1522e92df56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:46.699254 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513acbf2-402c-43b3-bd55-a1522e92df56 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:46.709402 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1254/2539] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:26:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:26:47.189446 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48132c1-b8c5-4bc9-a60b-19a04e798082 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:47.192097 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-b48132c1-b8c5-4bc9-a60b-19a04e798082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:47.209702 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48132c1-b8c5-4bc9-a60b-19a04e798082 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:47.216562 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215bb12b-27eb-4c29-b738-508d9db2b71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:47.217318 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215bb12b-27eb-4c29-b738-508d9db2b71d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:47.217697 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215bb12b-27eb-4c29-b738-508d9db2b71d admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:47.218090 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215bb12b-27eb-4c29-b738-508d9db2b71d admin admin] PATH_INFO: `/v3/projects/74f076c1966040abb26fc3641f4e164f` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:47.219930 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215bb12b-27eb-4c29-b738-508d9db2b71d admin admin] RBAC: Authorizing `identity:delete_project(project_id=74f076c1966040abb26fc3641f4e164f)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:47.222511 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215bb12b-27eb-4c29-b738-508d9db2b71d admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:47.328043 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:26:47.331504 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-215bb12b-27eb-4c29-b738-508d9db2b71d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74f076c1966040abb26fc3641f4e164f', 'request_id': 'req-215bb12b-27eb-4c29-b738-508d9db2b71d'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:47.337878 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-215bb12b-27eb-4c29-b738-508d9db2b71d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74f076c1966040abb26fc3641f4e164f', 'request_id': 'req-215bb12b-27eb-4c29-b738-508d9db2b71d'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:47.392077 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-215bb12b-27eb-4c29-b738-508d9db2b71d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74f076c1966040abb26fc3641f4e164f', 'request_id': 'req-215bb12b-27eb-4c29-b738-508d9db2b71d'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:47.408231 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-215bb12b-27eb-4c29-b738-508d9db2b71d admin admin] The token cache is being invalidate because project 74f076c1966040abb26fc3641f4e164f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:26:47.408800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-215bb12b-27eb-4c29-b738-508d9db2b71d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:47.409442 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-215bb12b-27eb-4c29-b738-508d9db2b71d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:47.411750 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1286/2540] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:26:47 2022] DELETE /identity/v3/projects/74f076c1966040abb26fc3641f4e164f => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:26:47.416862 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e91b38-4090-45e0-a4c4-af72b7ed7bf1 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:47.417807 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e91b38-4090-45e0-a4c4-af72b7ed7bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:47.417971 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e91b38-4090-45e0-a4c4-af72b7ed7bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:47.438306 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-83e91b38-4090-45e0-a4c4-af72b7ed7bf1 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:26:47.439606 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-83e91b38-4090-45e0-a4c4-af72b7ed7bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:47.543340 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1255/2541] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:26:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:26:47.550995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2a5dd-80f5-4e55-9e45-67d1bdeb7338 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:47.551454 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2a5dd-80f5-4e55-9e45-67d1bdeb7338 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:47.552542 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2a5dd-80f5-4e55-9e45-67d1bdeb7338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:47.584545 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-a0a2a5dd-80f5-4e55-9e45-67d1bdeb7338 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:26:47.586092 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-a0a2a5dd-80f5-4e55-9e45-67d1bdeb7338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:47.595143 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1287/2542] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:26:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:26:47.600575 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e91b38-4090-45e0-a4c4-af72b7ed7bf1 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:47.611713 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e91b38-4090-45e0-a4c4-af72b7ed7bf1 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:47.616970 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7841c9-55be-4801-93c7-b3151ab54072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:47.617591 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7841c9-55be-4801-93c7-b3151ab54072 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:47.617869 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7841c9-55be-4801-93c7-b3151ab54072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:47.618275 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7841c9-55be-4801-93c7-b3151ab54072 admin admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:47.619035 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7841c9-55be-4801-93c7-b3151ab54072 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:26:47.619289 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7841c9-55be-4801-93c7-b3151ab54072 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:47.621580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7841c9-55be-4801-93c7-b3151ab54072 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:47.636242 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1256/2543] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:26:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:26:47.640932 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2a5dd-80f5-4e55-9e45-67d1bdeb7338 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:47.657192 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2a5dd-80f5-4e55-9e45-67d1bdeb7338 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:47.657850 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9cf3d-8436-4868-becf-8c9749a4dcca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:47.658671 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9cf3d-8436-4868-becf-8c9749a4dcca admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:47.658977 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9cf3d-8436-4868-becf-8c9749a4dcca admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:47.659261 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9cf3d-8436-4868-becf-8c9749a4dcca admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:47.659991 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9cf3d-8436-4868-becf-8c9749a4dcca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:47.661950 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9cf3d-8436-4868-becf-8c9749a4dcca admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:47.763471 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1288/2544] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:26:47 2022] POST /identity/v3/projects => generated 375 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:26:47.765473 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7841c9-55be-4801-93c7-b3151ab54072 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:47.771622 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:26:47.780298 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7841c9-55be-4801-93c7-b3151ab54072 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:47.842425 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bbf2ad-2af3-4962-b507-8b3587d0f3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:47.843045 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bbf2ad-2af3-4962-b507-8b3587d0f3ea admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:47.843328 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bbf2ad-2af3-4962-b507-8b3587d0f3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:47.843655 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bbf2ad-2af3-4962-b507-8b3587d0f3ea admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:47.844340 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bbf2ad-2af3-4962-b507-8b3587d0f3ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:47.846266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bbf2ad-2af3-4962-b507-8b3587d0f3ea admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:47.942505 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1257/2545] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:26:47 2022] POST /identity/v3/users => generated 288 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:26:47.946863 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9cf3d-8436-4868-becf-8c9749a4dcca admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:47.965692 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9cf3d-8436-4868-becf-8c9749a4dcca admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:47.968684 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d582b3-d9c4-4746-b1e2-6e85049be037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:47.969362 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d582b3-d9c4-4746-b1e2-6e85049be037 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:47.969643 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d582b3-d9c4-4746-b1e2-6e85049be037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:47.969920 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d582b3-d9c4-4746-b1e2-6e85049be037 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:47.970557 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d582b3-d9c4-4746-b1e2-6e85049be037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:47.972308 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d582b3-d9c4-4746-b1e2-6e85049be037 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:47.984162 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1289/2546] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:26:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:26:47.988502 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bbf2ad-2af3-4962-b507-8b3587d0f3ea admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:48.000019 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bbf2ad-2af3-4962-b507-8b3587d0f3ea admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:48.005599 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4618b26-2953-428d-bf8c-38e498a04dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:48.006218 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4618b26-2953-428d-bf8c-38e498a04dee admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:48.006501 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4618b26-2953-428d-bf8c-38e498a04dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:48.006784 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4618b26-2953-428d-bf8c-38e498a04dee admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:48.007463 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4618b26-2953-428d-bf8c-38e498a04dee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:48.009079 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4618b26-2953-428d-bf8c-38e498a04dee admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:48.021026 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1258/2547] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:26:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:26:48.025320 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d582b3-d9c4-4746-b1e2-6e85049be037 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:48.035655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d582b3-d9c4-4746-b1e2-6e85049be037 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:48.040379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ff33c-2037-4cb7-95ba-a9a87b773c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:48.041029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ff33c-2037-4cb7-95ba-a9a87b773c85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:48.041314 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ff33c-2037-4cb7-95ba-a9a87b773c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:48.041594 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ff33c-2037-4cb7-95ba-a9a87b773c85 admin admin] PATH_INFO: `/v3/projects/00b03fe7e2354b919129088ef447ce2e/users/a24d95a983f641288a4883ffcfb931cc/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:48.056888 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109ff33c-2037-4cb7-95ba-a9a87b773c85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00b03fe7e2354b919129088ef447ce2e, user_id=a24d95a983f641288a4883ffcfb931cc, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:48.060173 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109ff33c-2037-4cb7-95ba-a9a87b773c85 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:48.166228 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1290/2548] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:26:48 2022] PUT /identity/v3/projects/00b03fe7e2354b919129088ef447ce2e/users/a24d95a983f641288a4883ffcfb931cc/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:26:48.177647 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0a336-c162-4287-9db1-6413bd7d7c5a None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:48.177940 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0a336-c162-4287-9db1-6413bd7d7c5a None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:48.178238 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0a336-c162-4287-9db1-6413bd7d7c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:48.207400 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-5be0a336-c162-4287-9db1-6413bd7d7c5a None None] MFA Rules not processed for user `a24d95a983f641288a4883ffcfb931cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:26:48.208692 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-5be0a336-c162-4287-9db1-6413bd7d7c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:48.286446 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1259/2549] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:26:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:26:48.298812 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ff33c-2037-4cb7-95ba-a9a87b773c85 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:48.301397 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-109ff33c-2037-4cb7-95ba-a9a87b773c85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:48.362223 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ff33c-2037-4cb7-95ba-a9a87b773c85 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:48.366002 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e3f0b-f73f-4a3d-8278-1ee22aafac26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:48.366850 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e3f0b-f73f-4a3d-8278-1ee22aafac26 service cinder] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:48.366959 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e3f0b-f73f-4a3d-8278-1ee22aafac26 service cinder] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:48.367184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e3f0b-f73f-4a3d-8278-1ee22aafac26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:48.376039 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e3f0b-f73f-4a3d-8278-1ee22aafac26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:48.377477 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e3f0b-f73f-4a3d-8278-1ee22aafac26 service cinder] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:48.387818 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1291/2550] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:26:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:26:49.371850 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be0a336-c162-4287-9db1-6413bd7d7c5a None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:49.373911 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-5be0a336-c162-4287-9db1-6413bd7d7c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:49.446120 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be0a336-c162-4287-9db1-6413bd7d7c5a None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:49.451333 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9204fce-d07f-4d45-9bba-23a6bf7c0c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:49.452023 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9204fce-d07f-4d45-9bba-23a6bf7c0c42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:49.456586 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9204fce-d07f-4d45-9bba-23a6bf7c0c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:49.456586 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9204fce-d07f-4d45-9bba-23a6bf7c0c42 admin admin] PATH_INFO: `/v3/users/b5113b1a77fd4b5da175b29b40d2b636` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:49.456586 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9204fce-d07f-4d45-9bba-23a6bf7c0c42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5113b1a77fd4b5da175b29b40d2b636)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:49.456586 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9204fce-d07f-4d45-9bba-23a6bf7c0c42 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:49.468694 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:26:49.656751 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-f9204fce-d07f-4d45-9bba-23a6bf7c0c42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5113b1a77fd4b5da175b29b40d2b636'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:49.657457 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-f9204fce-d07f-4d45-9bba-23a6bf7c0c42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5113b1a77fd4b5da175b29b40d2b636'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:49.658064 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-f9204fce-d07f-4d45-9bba-23a6bf7c0c42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5113b1a77fd4b5da175b29b40d2b636'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:49.665135 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-f9204fce-d07f-4d45-9bba-23a6bf7c0c42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5113b1a77fd4b5da175b29b40d2b636'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:49.679355 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-f9204fce-d07f-4d45-9bba-23a6bf7c0c42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5113b1a77fd4b5da175b29b40d2b636'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:49.772352 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1260/2551] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:26:49 2022] DELETE /identity/v3/users/b5113b1a77fd4b5da175b29b40d2b636 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:26:50.088552 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e3f0b-f73f-4a3d-8278-1ee22aafac26 service cinder] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:50.102361 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e3f0b-f73f-4a3d-8278-1ee22aafac26 service cinder] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:50.160351 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a1b681-79e6-49ae-8729-39e910ac26d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:50.160981 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1b681-79e6-49ae-8729-39e910ac26d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:50.161270 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1b681-79e6-49ae-8729-39e910ac26d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:50.161545 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1b681-79e6-49ae-8729-39e910ac26d4 admin admin] PATH_INFO: `/v3/projects/b1f3e22eb01045f299926f67cd36b91d` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:50.162806 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a1b681-79e6-49ae-8729-39e910ac26d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1f3e22eb01045f299926f67cd36b91d)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:50.164950 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a1b681-79e6-49ae-8729-39e910ac26d4 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:50.181972 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:26:50.290077 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-31a1b681-79e6-49ae-8729-39e910ac26d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1f3e22eb01045f299926f67cd36b91d', 'request_id': 'req-31a1b681-79e6-49ae-8729-39e910ac26d4'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:50.296125 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-31a1b681-79e6-49ae-8729-39e910ac26d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1f3e22eb01045f299926f67cd36b91d', 'request_id': 'req-31a1b681-79e6-49ae-8729-39e910ac26d4'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:50.363130 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-31a1b681-79e6-49ae-8729-39e910ac26d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1f3e22eb01045f299926f67cd36b91d', 'request_id': 'req-31a1b681-79e6-49ae-8729-39e910ac26d4'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:50.376644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-31a1b681-79e6-49ae-8729-39e910ac26d4 admin admin] The token cache is being invalidate because project b1f3e22eb01045f299926f67cd36b91d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:26:50.377111 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-31a1b681-79e6-49ae-8729-39e910ac26d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:50.377781 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-31a1b681-79e6-49ae-8729-39e910ac26d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:50.379288 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1292/2552] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:26:50 2022] DELETE /identity/v3/projects/b1f3e22eb01045f299926f67cd36b91d => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:26:50.384622 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe781ab-809b-4341-962b-bbd685eacb67 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:50.385517 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe781ab-809b-4341-962b-bbd685eacb67 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:50.386366 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe781ab-809b-4341-962b-bbd685eacb67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:50.407334 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-ebe781ab-809b-4341-962b-bbd685eacb67 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:26:50.408655 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-ebe781ab-809b-4341-962b-bbd685eacb67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:50.469603 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1261/2553] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:26:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:26:50.477586 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e1e116-3b2b-444f-817e-ff0782c35af1 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:50.477882 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e1e116-3b2b-444f-817e-ff0782c35af1 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:50.478198 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e1e116-3b2b-444f-817e-ff0782c35af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:50.498447 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-39e1e116-3b2b-444f-817e-ff0782c35af1 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:26:50.499696 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-39e1e116-3b2b-444f-817e-ff0782c35af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:50.507840 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1293/2554] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:26:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:26:50.512639 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe781ab-809b-4341-962b-bbd685eacb67 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:50.522330 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe781ab-809b-4341-962b-bbd685eacb67 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:50.527639 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3491fc4-d329-4d50-84a7-683994d749d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:50.528243 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3491fc4-d329-4d50-84a7-683994d749d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:50.528530 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3491fc4-d329-4d50-84a7-683994d749d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:50.528806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3491fc4-d329-4d50-84a7-683994d749d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:50.529460 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3491fc4-d329-4d50-84a7-683994d749d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:26:50.529769 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3491fc4-d329-4d50-84a7-683994d749d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:50.531368 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3491fc4-d329-4d50-84a7-683994d749d5 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:50.542919 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1262/2555] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:26:50.547484 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e1e116-3b2b-444f-817e-ff0782c35af1 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:50.557355 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e1e116-3b2b-444f-817e-ff0782c35af1 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:50.562943 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818ccd6-3a7e-4a3b-8b9f-34d03ac07344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:50.563645 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818ccd6-3a7e-4a3b-8b9f-34d03ac07344 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:50.564038 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818ccd6-3a7e-4a3b-8b9f-34d03ac07344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:50.564229 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818ccd6-3a7e-4a3b-8b9f-34d03ac07344 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:50.564895 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9818ccd6-3a7e-4a3b-8b9f-34d03ac07344 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:50.566799 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9818ccd6-3a7e-4a3b-8b9f-34d03ac07344 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:50.709884 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1294/2556] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:26:50 2022] POST /identity/v3/projects => generated 375 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:26:50.711261 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3491fc4-d329-4d50-84a7-683994d749d5 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:50.721775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3491fc4-d329-4d50-84a7-683994d749d5 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:50.780440 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb74c5a5-887f-4ff3-9990-7436339e1502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:50.781060 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74c5a5-887f-4ff3-9990-7436339e1502 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:50.781341 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74c5a5-887f-4ff3-9990-7436339e1502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:50.781617 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74c5a5-887f-4ff3-9990-7436339e1502 admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:50.782253 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb74c5a5-887f-4ff3-9990-7436339e1502 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:50.784239 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb74c5a5-887f-4ff3-9990-7436339e1502 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:50.894116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1263/2557] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:26:50 2022] POST /identity/v3/users => generated 288 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:26:50.894228 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818ccd6-3a7e-4a3b-8b9f-34d03ac07344 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:50.905012 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818ccd6-3a7e-4a3b-8b9f-34d03ac07344 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:50.909704 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf96ee-a246-42a8-82ea-77efb19108a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:50.910301 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf96ee-a246-42a8-82ea-77efb19108a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:50.910578 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf96ee-a246-42a8-82ea-77efb19108a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:50.910878 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf96ee-a246-42a8-82ea-77efb19108a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:50.911552 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cf96ee-a246-42a8-82ea-77efb19108a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:50.913129 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cf96ee-a246-42a8-82ea-77efb19108a3 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:50.924257 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1295/2558] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 12:26:50.933478 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb74c5a5-887f-4ff3-9990-7436339e1502 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:50.944976 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb74c5a5-887f-4ff3-9990-7436339e1502 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:50.954485 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb81d7d9-7985-4244-865a-d16abc102f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:50.955776 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb81d7d9-7985-4244-865a-d16abc102f5a admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:50.956065 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb81d7d9-7985-4244-865a-d16abc102f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:50.956341 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb81d7d9-7985-4244-865a-d16abc102f5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:50.957000 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb81d7d9-7985-4244-865a-d16abc102f5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:50.958605 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb81d7d9-7985-4244-865a-d16abc102f5a admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:50.961170 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:26:50.972310 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1264/2559] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:26:50.976909 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf96ee-a246-42a8-82ea-77efb19108a3 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:50.988517 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf96ee-a246-42a8-82ea-77efb19108a3 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:50.993304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ef355-1ed5-4095-84be-d7ec9c951aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:50.993959 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ef355-1ed5-4095-84be-d7ec9c951aec admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:50.994253 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ef355-1ed5-4095-84be-d7ec9c951aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:50.994540 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ef355-1ed5-4095-84be-d7ec9c951aec admin admin] PATH_INFO: `/v3/projects/5218505298ee478281862c26f50168e2/users/dcd459306c854e80a4bbe0d40f5ae75c/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:51.009970 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578ef355-1ed5-4095-84be-d7ec9c951aec admin admin] RBAC: Authorizing `identity:create_grant(project_id=5218505298ee478281862c26f50168e2, user_id=dcd459306c854e80a4bbe0d40f5ae75c, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:51.013380 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578ef355-1ed5-4095-84be-d7ec9c951aec admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:51.071973 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1296/2560] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:26:50 2022] PUT /identity/v3/projects/5218505298ee478281862c26f50168e2/users/dcd459306c854e80a4bbe0d40f5ae75c/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:26:51.083354 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570e529-632d-4739-b8a0-ff6f95b499a5 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:51.083765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570e529-632d-4739-b8a0-ff6f95b499a5 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:51.084139 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570e529-632d-4739-b8a0-ff6f95b499a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:51.112862 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-3570e529-632d-4739-b8a0-ff6f95b499a5 None None] MFA Rules not processed for user `dcd459306c854e80a4bbe0d40f5ae75c`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:26:51.114075 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-3570e529-632d-4739-b8a0-ff6f95b499a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:51.178274 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1265/2561] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:26:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:26:51.190368 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ef355-1ed5-4095-84be-d7ec9c951aec admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:51.192127 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-578ef355-1ed5-4095-84be-d7ec9c951aec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:51.270488 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ef355-1ed5-4095-84be-d7ec9c951aec admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:51.276111 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b48bcd-20fa-4009-b0a0-870481f574ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:51.276936 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b48bcd-20fa-4009-b0a0-870481f574ac service nova] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:51.277199 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b48bcd-20fa-4009-b0a0-870481f574ac service nova] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:51.277471 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b48bcd-20fa-4009-b0a0-870481f574ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:51.285562 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b48bcd-20fa-4009-b0a0-870481f574ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:51.287281 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b48bcd-20fa-4009-b0a0-870481f574ac service nova] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:51.296809 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1297/2562] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:26:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:26:51.507180 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3570e529-632d-4739-b8a0-ff6f95b499a5 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:51.568002 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3570e529-632d-4739-b8a0-ff6f95b499a5 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:51.572452 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a46b7b-abab-4590-9481-c95cb2ececa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:51.573071 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a46b7b-abab-4590-9481-c95cb2ececa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:51.573456 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a46b7b-abab-4590-9481-c95cb2ececa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:51.573733 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a46b7b-abab-4590-9481-c95cb2ececa1 admin admin] PATH_INFO: `/v3/users/dcd459306c854e80a4bbe0d40f5ae75c` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:51.574843 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a46b7b-abab-4590-9481-c95cb2ececa1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcd459306c854e80a4bbe0d40f5ae75c)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:51.576851 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a46b7b-abab-4590-9481-c95cb2ececa1 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:51.808995 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-f0a46b7b-abab-4590-9481-c95cb2ececa1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcd459306c854e80a4bbe0d40f5ae75c'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:51.809673 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-f0a46b7b-abab-4590-9481-c95cb2ececa1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcd459306c854e80a4bbe0d40f5ae75c'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:51.810348 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-f0a46b7b-abab-4590-9481-c95cb2ececa1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcd459306c854e80a4bbe0d40f5ae75c'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:51.817924 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-f0a46b7b-abab-4590-9481-c95cb2ececa1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcd459306c854e80a4bbe0d40f5ae75c'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:51.830654 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-f0a46b7b-abab-4590-9481-c95cb2ececa1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcd459306c854e80a4bbe0d40f5ae75c'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:51.907908 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1266/2563] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:26:51 2022] DELETE /identity/v3/users/dcd459306c854e80a4bbe0d40f5ae75c => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:26:51.909623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084b493-b2e0-48d2-b573-9d97291e2c64 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:51.909989 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084b493-b2e0-48d2-b573-9d97291e2c64 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:51.910273 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084b493-b2e0-48d2-b573-9d97291e2c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:51.930799 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-a084b493-b2e0-48d2-b573-9d97291e2c64 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:26:51.932057 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-a084b493-b2e0-48d2-b573-9d97291e2c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:51.953118 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:26:51.993546 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1298/2564] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:26:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:26:52.003494 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a46b7b-abab-4590-9481-c95cb2ececa1 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:52.004919 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-f0a46b7b-abab-4590-9481-c95cb2ececa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:52.012092 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:26:52.063361 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a46b7b-abab-4590-9481-c95cb2ececa1 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:52.068005 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9a54b-f70b-4b04-865f-5c776a994e69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:52.068621 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9a54b-f70b-4b04-865f-5c776a994e69 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:52.068908 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9a54b-f70b-4b04-865f-5c776a994e69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:52.069184 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9a54b-f70b-4b04-865f-5c776a994e69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:52.078920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9a54b-f70b-4b04-865f-5c776a994e69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:52.080385 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9a54b-f70b-4b04-865f-5c776a994e69 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:52.090133 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1267/2565] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:26:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:26:52.485134 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a084b493-b2e0-48d2-b573-9d97291e2c64 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:52.486748 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-a084b493-b2e0-48d2-b573-9d97291e2c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:52.499599 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a084b493-b2e0-48d2-b573-9d97291e2c64 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:52.505133 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b3b59-8a3f-492b-acab-a9cc522dd19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:52.505133 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b3b59-8a3f-492b-acab-a9cc522dd19e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:52.505351 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b3b59-8a3f-492b-acab-a9cc522dd19e admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:52.505595 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b3b59-8a3f-492b-acab-a9cc522dd19e admin admin] PATH_INFO: `/v3/projects/5218505298ee478281862c26f50168e2` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:52.506824 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18b3b59-8a3f-492b-acab-a9cc522dd19e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5218505298ee478281862c26f50168e2)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:52.508781 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18b3b59-8a3f-492b-acab-a9cc522dd19e admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:52.639176 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-f18b3b59-8a3f-492b-acab-a9cc522dd19e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5218505298ee478281862c26f50168e2', 'request_id': 'req-f18b3b59-8a3f-492b-acab-a9cc522dd19e'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:52.644643 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-f18b3b59-8a3f-492b-acab-a9cc522dd19e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5218505298ee478281862c26f50168e2', 'request_id': 'req-f18b3b59-8a3f-492b-acab-a9cc522dd19e'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:52.769221 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-f18b3b59-8a3f-492b-acab-a9cc522dd19e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5218505298ee478281862c26f50168e2', 'request_id': 'req-f18b3b59-8a3f-492b-acab-a9cc522dd19e'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:52.781640 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-f18b3b59-8a3f-492b-acab-a9cc522dd19e admin admin] The token cache is being invalidate because project 5218505298ee478281862c26f50168e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:26:52.781935 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-f18b3b59-8a3f-492b-acab-a9cc522dd19e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:52.782604 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-f18b3b59-8a3f-492b-acab-a9cc522dd19e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:52.784137 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1299/2566] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:26:52 2022] DELETE /identity/v3/projects/5218505298ee478281862c26f50168e2 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:26:52.789242 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a421f68-5000-49d6-90b2-f844345d8faf None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:52.789757 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a421f68-5000-49d6-90b2-f844345d8faf None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:52.789861 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a421f68-5000-49d6-90b2-f844345d8faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:52.810466 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-2a421f68-5000-49d6-90b2-f844345d8faf None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:26:52.811708 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-2a421f68-5000-49d6-90b2-f844345d8faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:52.871691 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1268/2567] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:26:52.879577 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2276b-88ee-465e-8416-5f45c591e734 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:52.879873 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2276b-88ee-465e-8416-5f45c591e734 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:52.884689 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2276b-88ee-465e-8416-5f45c591e734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:52.900210 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-83b2276b-88ee-465e-8416-5f45c591e734 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:26:52.901906 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-83b2276b-88ee-465e-8416-5f45c591e734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:52.909959 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1300/2568] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:26:52.914201 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a421f68-5000-49d6-90b2-f844345d8faf None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:52.924491 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a421f68-5000-49d6-90b2-f844345d8faf None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:52.929421 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03123a61-7ee9-452b-b1ff-b18358dc2fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:52.930039 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03123a61-7ee9-452b-b1ff-b18358dc2fc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:52.930318 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03123a61-7ee9-452b-b1ff-b18358dc2fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:52.930633 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03123a61-7ee9-452b-b1ff-b18358dc2fc2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:52.931226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03123a61-7ee9-452b-b1ff-b18358dc2fc2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:26:52.931580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03123a61-7ee9-452b-b1ff-b18358dc2fc2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:52.933162 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03123a61-7ee9-452b-b1ff-b18358dc2fc2 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:52.945182 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1269/2569] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:26:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:26:52.949494 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b2276b-88ee-465e-8416-5f45c591e734 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:52.959096 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b2276b-88ee-465e-8416-5f45c591e734 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:52.964658 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ebffd-c1c1-4f41-a6bf-c12dbe1fbed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:52.965263 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ebffd-c1c1-4f41-a6bf-c12dbe1fbed4 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:52.965563 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ebffd-c1c1-4f41-a6bf-c12dbe1fbed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:52.965820 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ebffd-c1c1-4f41-a6bf-c12dbe1fbed4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:52.966447 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ebffd-c1c1-4f41-a6bf-c12dbe1fbed4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:52.968511 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ebffd-c1c1-4f41-a6bf-c12dbe1fbed4 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:53.053654 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1301/2570] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:26:52 2022] POST /identity/v3/projects => generated 405 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:26:53.057750 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03123a61-7ee9-452b-b1ff-b18358dc2fc2 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:53.067655 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03123a61-7ee9-452b-b1ff-b18358dc2fc2 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:53.123560 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d844f0f6-95db-4522-8177-078ce2668941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:53.124172 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844f0f6-95db-4522-8177-078ce2668941 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:53.124453 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844f0f6-95db-4522-8177-078ce2668941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:53.124776 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844f0f6-95db-4522-8177-078ce2668941 admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:53.125358 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d844f0f6-95db-4522-8177-078ce2668941 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:53.127273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d844f0f6-95db-4522-8177-078ce2668941 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:53.203311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1270/2571] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:26:53 2022] POST /identity/v3/users => generated 303 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:26:53.207327 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ebffd-c1c1-4f41-a6bf-c12dbe1fbed4 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:53.218305 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ebffd-c1c1-4f41-a6bf-c12dbe1fbed4 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:53.222938 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f889b1da-4960-42e2-9cd0-3e87647cea2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:53.223574 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f889b1da-4960-42e2-9cd0-3e87647cea2f admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:53.223861 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f889b1da-4960-42e2-9cd0-3e87647cea2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:53.224136 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f889b1da-4960-42e2-9cd0-3e87647cea2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:53.229296 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f889b1da-4960-42e2-9cd0-3e87647cea2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:53.229296 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f889b1da-4960-42e2-9cd0-3e87647cea2f admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:53.237007 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1302/2572] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:26:53.240947 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d844f0f6-95db-4522-8177-078ce2668941 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:53.251326 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d844f0f6-95db-4522-8177-078ce2668941 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:53.256273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad624c-3599-4048-9ac8-0dd8a8bcc721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:53.256882 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad624c-3599-4048-9ac8-0dd8a8bcc721 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:53.257170 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad624c-3599-4048-9ac8-0dd8a8bcc721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:53.257443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad624c-3599-4048-9ac8-0dd8a8bcc721 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:53.258088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afad624c-3599-4048-9ac8-0dd8a8bcc721 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:53.259807 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afad624c-3599-4048-9ac8-0dd8a8bcc721 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:53.270574 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1271/2573] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:26:53.274747 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f889b1da-4960-42e2-9cd0-3e87647cea2f admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:53.285319 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f889b1da-4960-42e2-9cd0-3e87647cea2f admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:53.292791 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e7cd6-b659-4a65-8877-67afc2fc5a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:53.293449 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e7cd6-b659-4a65-8877-67afc2fc5a52 admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:53.293734 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e7cd6-b659-4a65-8877-67afc2fc5a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:53.294012 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e7cd6-b659-4a65-8877-67afc2fc5a52 admin admin] PATH_INFO: `/v3/projects/0046e0ff546d408bac156022880fbcae/users/4f9f90e56b53452ba699b6999efe2b61/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:53.309176 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825e7cd6-b659-4a65-8877-67afc2fc5a52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0046e0ff546d408bac156022880fbcae, user_id=4f9f90e56b53452ba699b6999efe2b61, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:53.311928 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825e7cd6-b659-4a65-8877-67afc2fc5a52 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:53.384569 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1303/2574] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:26:53 2022] PUT /identity/v3/projects/0046e0ff546d408bac156022880fbcae/users/4f9f90e56b53452ba699b6999efe2b61/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:26:53.390088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6bc05-e71a-429c-8813-c5ab898dad62 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:53.390401 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6bc05-e71a-429c-8813-c5ab898dad62 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:53.390682 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6bc05-e71a-429c-8813-c5ab898dad62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:53.410224 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-c3a6bc05-e71a-429c-8813-c5ab898dad62 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:26:53.411464 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-c3a6bc05-e71a-429c-8813-c5ab898dad62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:53.419752 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:26:53.472373 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1272/2575] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:26:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:26:53.482316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e7cd6-b659-4a65-8877-67afc2fc5a52 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:53.484070 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-825e7cd6-b659-4a65-8877-67afc2fc5a52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:53.530446 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:26:53.540970 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e7cd6-b659-4a65-8877-67afc2fc5a52 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:53.544618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c5cb2-6e54-48f4-af66-033ef3a160b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:53.545226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c5cb2-6e54-48f4-af66-033ef3a160b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:53.545511 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c5cb2-6e54-48f4-af66-033ef3a160b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:53.545788 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c5cb2-6e54-48f4-af66-033ef3a160b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:53.554668 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19c5cb2-6e54-48f4-af66-033ef3a160b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:53.556130 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19c5cb2-6e54-48f4-af66-033ef3a160b7 service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:53.565452 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1304/2576] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:26:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:26:54.916103 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80200529-e52a-4db3-b990-474df542339f None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:54.916634 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80200529-e52a-4db3-b990-474df542339f None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:54.916634 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80200529-e52a-4db3-b990-474df542339f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:54.945912 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-80200529-e52a-4db3-b990-474df542339f None None] MFA Rules not processed for user `4f9f90e56b53452ba699b6999efe2b61`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:26:54.947053 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-80200529-e52a-4db3-b990-474df542339f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:55.021440 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1273/2577] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:26:54 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:26:55.022616 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c5cb2-6e54-48f4-af66-033ef3a160b7 service neutron] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:55.038503 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c5cb2-6e54-48f4-af66-033ef3a160b7 service neutron] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:55.043026 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4523cfe-9bd6-49f5-976d-72fc8bfc40fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:55.043680 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4523cfe-9bd6-49f5-976d-72fc8bfc40fd admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:55.043968 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4523cfe-9bd6-49f5-976d-72fc8bfc40fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:55.044245 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4523cfe-9bd6-49f5-976d-72fc8bfc40fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:55.044879 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4523cfe-9bd6-49f5-976d-72fc8bfc40fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:55.046779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4523cfe-9bd6-49f5-976d-72fc8bfc40fd admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:55.137382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1305/2578] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:26:55 2022] POST /identity/v3/projects => generated 403 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:26:55.141555 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80200529-e52a-4db3-b990-474df542339f None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:55.153137 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80200529-e52a-4db3-b990-474df542339f None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:55.213854 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6df6f3-e0a3-4336-b751-99eb3a365b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:55.217609 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6df6f3-e0a3-4336-b751-99eb3a365b66 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:55.217609 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6df6f3-e0a3-4336-b751-99eb3a365b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:55.217609 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6df6f3-e0a3-4336-b751-99eb3a365b66 admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:55.217609 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6df6f3-e0a3-4336-b751-99eb3a365b66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:55.217992 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6df6f3-e0a3-4336-b751-99eb3a365b66 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:55.386840 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1274/2579] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:26:55 2022] POST /identity/v3/users => generated 308 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:26:55.391074 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4523cfe-9bd6-49f5-976d-72fc8bfc40fd admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:55.401557 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4523cfe-9bd6-49f5-976d-72fc8bfc40fd admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:55.406225 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf18c9e-f701-4aeb-a22f-59a2ee8b0e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:55.406843 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf18c9e-f701-4aeb-a22f-59a2ee8b0e8a admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:55.407142 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf18c9e-f701-4aeb-a22f-59a2ee8b0e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:55.407439 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf18c9e-f701-4aeb-a22f-59a2ee8b0e8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:55.408120 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf18c9e-f701-4aeb-a22f-59a2ee8b0e8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:55.409763 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf18c9e-f701-4aeb-a22f-59a2ee8b0e8a admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:55.429474 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1306/2580] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:26:55.433991 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6df6f3-e0a3-4336-b751-99eb3a365b66 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:55.451018 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6df6f3-e0a3-4336-b751-99eb3a365b66 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:55.451484 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358f214d-1cab-42e2-9c1c-0994623f3abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:55.452153 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f214d-1cab-42e2-9c1c-0994623f3abc admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:55.452431 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f214d-1cab-42e2-9c1c-0994623f3abc admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:55.452759 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f214d-1cab-42e2-9c1c-0994623f3abc admin admin] PATH_INFO: `/v3/projects/2912773901474c228ab3bd0b8104c99c/users/1f9e61e222c44c2aad15c977356829c1/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:55.468058 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358f214d-1cab-42e2-9c1c-0994623f3abc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2912773901474c228ab3bd0b8104c99c, user_id=1f9e61e222c44c2aad15c977356829c1, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:55.470636 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358f214d-1cab-42e2-9c1c-0994623f3abc admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:55.547030 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1275/2581] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:26:55 2022] PUT /identity/v3/projects/2912773901474c228ab3bd0b8104c99c/users/1f9e61e222c44c2aad15c977356829c1/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:26:55.552081 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf18c9e-f701-4aeb-a22f-59a2ee8b0e8a admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:55.563929 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf18c9e-f701-4aeb-a22f-59a2ee8b0e8a admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:55.626328 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09efcbf3-477d-4e7f-affc-a9db04f90d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:55.626941 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09efcbf3-477d-4e7f-affc-a9db04f90d8e admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:55.627936 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09efcbf3-477d-4e7f-affc-a9db04f90d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:55.627936 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09efcbf3-477d-4e7f-affc-a9db04f90d8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:55.628211 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09efcbf3-477d-4e7f-affc-a9db04f90d8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:55.629770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09efcbf3-477d-4e7f-affc-a9db04f90d8e admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:55.640357 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1307/2582] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 12:26:55.644601 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358f214d-1cab-42e2-9c1c-0994623f3abc admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:55.657844 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358f214d-1cab-42e2-9c1c-0994623f3abc admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:55.662629 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76441b-08e9-45de-98f7-ec67e95a6288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:55.663293 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76441b-08e9-45de-98f7-ec67e95a6288 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:55.663608 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76441b-08e9-45de-98f7-ec67e95a6288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:55.663888 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76441b-08e9-45de-98f7-ec67e95a6288 admin admin] PATH_INFO: `/v3/projects/2912773901474c228ab3bd0b8104c99c/users/1f9e61e222c44c2aad15c977356829c1/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:55.665884 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76441b-08e9-45de-98f7-ec67e95a6288 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2912773901474c228ab3bd0b8104c99c, user_id=1f9e61e222c44c2aad15c977356829c1, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:55.669234 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76441b-08e9-45de-98f7-ec67e95a6288 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:55.673879 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:26:55.680984 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1276/2583] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:26:55 2022] PUT /identity/v3/projects/2912773901474c228ab3bd0b8104c99c/users/1f9e61e222c44c2aad15c977356829c1/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:26:56.959658 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f663de0-dcbb-4ce4-ac36-0d3e60283221 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:56.960309 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f663de0-dcbb-4ce4-ac36-0d3e60283221 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:56.960309 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f663de0-dcbb-4ce4-ac36-0d3e60283221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:56.990597 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-6f663de0-dcbb-4ce4-ac36-0d3e60283221 None None] MFA Rules not processed for user `1f9e61e222c44c2aad15c977356829c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:26:56.991953 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-6f663de0-dcbb-4ce4-ac36-0d3e60283221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:57.066172 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1308/2584] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:26:56 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:26:58.399564 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76441b-08e9-45de-98f7-ec67e95a6288 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:58.465423 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76441b-08e9-45de-98f7-ec67e95a6288 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:58.470000 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd34d61-cd55-40a4-8b4e-1b78edf0b2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:58.470629 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd34d61-cd55-40a4-8b4e-1b78edf0b2d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:58.470914 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd34d61-cd55-40a4-8b4e-1b78edf0b2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:58.471196 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd34d61-cd55-40a4-8b4e-1b78edf0b2d2 admin admin] PATH_INFO: `/v3/users/4f9f90e56b53452ba699b6999efe2b61` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:58.472385 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd34d61-cd55-40a4-8b4e-1b78edf0b2d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f9f90e56b53452ba699b6999efe2b61)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:58.474342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd34d61-cd55-40a4-8b4e-1b78edf0b2d2 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:58.679191 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-8fd34d61-cd55-40a4-8b4e-1b78edf0b2d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f9f90e56b53452ba699b6999efe2b61'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:58.679898 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-8fd34d61-cd55-40a4-8b4e-1b78edf0b2d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f9f90e56b53452ba699b6999efe2b61'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:58.680511 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-8fd34d61-cd55-40a4-8b4e-1b78edf0b2d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f9f90e56b53452ba699b6999efe2b61'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:58.688874 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-8fd34d61-cd55-40a4-8b4e-1b78edf0b2d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f9f90e56b53452ba699b6999efe2b61'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:58.706591 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-8fd34d61-cd55-40a4-8b4e-1b78edf0b2d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f9f90e56b53452ba699b6999efe2b61'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:58.820783 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1277/2585] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:26:58 2022] DELETE /identity/v3/users/4f9f90e56b53452ba699b6999efe2b61 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:26:59.067756 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f663de0-dcbb-4ce4-ac36-0d3e60283221 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:59.080804 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f663de0-dcbb-4ce4-ac36-0d3e60283221 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:59.128551 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:26:59.153487 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff53e9b-bf7b-44df-9e92-f82d569b624d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:59.154120 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff53e9b-bf7b-44df-9e92-f82d569b624d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:59.154405 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff53e9b-bf7b-44df-9e92-f82d569b624d admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:59.154676 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff53e9b-bf7b-44df-9e92-f82d569b624d admin admin] PATH_INFO: `/v3/projects/0046e0ff546d408bac156022880fbcae` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:59.155860 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff53e9b-bf7b-44df-9e92-f82d569b624d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0046e0ff546d408bac156022880fbcae)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:59.158105 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff53e9b-bf7b-44df-9e92-f82d569b624d admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:59.255294 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-2ff53e9b-bf7b-44df-9e92-f82d569b624d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0046e0ff546d408bac156022880fbcae', 'request_id': 'req-2ff53e9b-bf7b-44df-9e92-f82d569b624d'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:59.262033 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-2ff53e9b-bf7b-44df-9e92-f82d569b624d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0046e0ff546d408bac156022880fbcae', 'request_id': 'req-2ff53e9b-bf7b-44df-9e92-f82d569b624d'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:59.351329 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-2ff53e9b-bf7b-44df-9e92-f82d569b624d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0046e0ff546d408bac156022880fbcae', 'request_id': 'req-2ff53e9b-bf7b-44df-9e92-f82d569b624d'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:59.364673 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-2ff53e9b-bf7b-44df-9e92-f82d569b624d admin admin] The token cache is being invalidate because project 0046e0ff546d408bac156022880fbcae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:26:59.364971 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-2ff53e9b-bf7b-44df-9e92-f82d569b624d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:59.365641 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-2ff53e9b-bf7b-44df-9e92-f82d569b624d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:59.372231 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1309/2586] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:26:59 2022] DELETE /identity/v3/projects/0046e0ff546d408bac156022880fbcae => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:26:59.378101 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd34d61-cd55-40a4-8b4e-1b78edf0b2d2 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:26:59.379525 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-8fd34d61-cd55-40a4-8b4e-1b78edf0b2d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:26:59.395715 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd34d61-cd55-40a4-8b4e-1b78edf0b2d2 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:26:59.451215 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cdb5be-b58f-4360-857a-767fc9532c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:26:59.451961 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdb5be-b58f-4360-857a-767fc9532c5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:26:59.452149 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdb5be-b58f-4360-857a-767fc9532c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:26:59.452420 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdb5be-b58f-4360-857a-767fc9532c5a admin admin] PATH_INFO: `/v3/users/1f9e61e222c44c2aad15c977356829c1` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:26:59.453563 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cdb5be-b58f-4360-857a-767fc9532c5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f9e61e222c44c2aad15c977356829c1)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:26:59.455506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cdb5be-b58f-4360-857a-767fc9532c5a admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:26:59.471734 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:26:59.658048 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-b6cdb5be-b58f-4360-857a-767fc9532c5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f9e61e222c44c2aad15c977356829c1'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:59.658742 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-b6cdb5be-b58f-4360-857a-767fc9532c5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f9e61e222c44c2aad15c977356829c1'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:59.659728 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-b6cdb5be-b58f-4360-857a-767fc9532c5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f9e61e222c44c2aad15c977356829c1'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:59.668208 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-b6cdb5be-b58f-4360-857a-767fc9532c5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f9e61e222c44c2aad15c977356829c1'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:59.681230 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-b6cdb5be-b58f-4360-857a-767fc9532c5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f9e61e222c44c2aad15c977356829c1'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:26:59.904565 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1278/2587] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:26:59 2022] DELETE /identity/v3/users/1f9e61e222c44c2aad15c977356829c1 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:27:00.163094 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff53e9b-bf7b-44df-9e92-f82d569b624d admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:00.164715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-2ff53e9b-bf7b-44df-9e92-f82d569b624d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:00.177775 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff53e9b-bf7b-44df-9e92-f82d569b624d admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:00.237197 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f8692-6780-4ad3-bd69-9ea397a4ea20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:00.237825 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f8692-6780-4ad3-bd69-9ea397a4ea20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:00.238108 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f8692-6780-4ad3-bd69-9ea397a4ea20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:00.238383 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f8692-6780-4ad3-bd69-9ea397a4ea20 admin admin] PATH_INFO: `/v3/projects/2912773901474c228ab3bd0b8104c99c` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:00.239531 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4f8692-6780-4ad3-bd69-9ea397a4ea20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2912773901474c228ab3bd0b8104c99c)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:00.241511 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4f8692-6780-4ad3-bd69-9ea397a4ea20 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:00.359067 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-fe4f8692-6780-4ad3-bd69-9ea397a4ea20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2912773901474c228ab3bd0b8104c99c', 'request_id': 'req-fe4f8692-6780-4ad3-bd69-9ea397a4ea20'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:00.365192 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-fe4f8692-6780-4ad3-bd69-9ea397a4ea20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2912773901474c228ab3bd0b8104c99c', 'request_id': 'req-fe4f8692-6780-4ad3-bd69-9ea397a4ea20'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:00.437558 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-fe4f8692-6780-4ad3-bd69-9ea397a4ea20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2912773901474c228ab3bd0b8104c99c', 'request_id': 'req-fe4f8692-6780-4ad3-bd69-9ea397a4ea20'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:00.439875 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:27:00.449597 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-fe4f8692-6780-4ad3-bd69-9ea397a4ea20 admin admin] The token cache is being invalidate because project 2912773901474c228ab3bd0b8104c99c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:27:00.449892 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-fe4f8692-6780-4ad3-bd69-9ea397a4ea20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:00.451138 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-fe4f8692-6780-4ad3-bd69-9ea397a4ea20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:00.452755 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1310/2588] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:27:00 2022] DELETE /identity/v3/projects/2912773901474c228ab3bd0b8104c99c => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:27:00.457892 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a8b70-be8c-4e03-97ca-8b703d5d487b None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:00.458492 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a8b70-be8c-4e03-97ca-8b703d5d487b None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:00.458492 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a8b70-be8c-4e03-97ca-8b703d5d487b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:00.478408 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-977a8b70-be8c-4e03-97ca-8b703d5d487b None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:00.479685 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-977a8b70-be8c-4e03-97ca-8b703d5d487b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:00.544093 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1279/2589] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:27:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:27:00.551958 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659157c-c527-43d0-a69c-54515e1dcac4 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:00.552255 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659157c-c527-43d0-a69c-54515e1dcac4 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:00.552541 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659157c-c527-43d0-a69c-54515e1dcac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:00.573300 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-6659157c-c527-43d0-a69c-54515e1dcac4 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:00.575144 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-6659157c-c527-43d0-a69c-54515e1dcac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:00.583361 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1311/2590] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:27:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:27:00.587847 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a8b70-be8c-4e03-97ca-8b703d5d487b None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:00.603683 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a8b70-be8c-4e03-97ca-8b703d5d487b None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:00.603683 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa3070-4644-45e2-94ad-ca8042a10de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:00.603948 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa3070-4644-45e2-94ad-ca8042a10de9 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:00.604237 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa3070-4644-45e2-94ad-ca8042a10de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:00.604513 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa3070-4644-45e2-94ad-ca8042a10de9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:00.605217 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fa3070-4644-45e2-94ad-ca8042a10de9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:27:00.605519 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fa3070-4644-45e2-94ad-ca8042a10de9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:00.607529 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fa3070-4644-45e2-94ad-ca8042a10de9 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:00.621209 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1280/2591] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:27:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:27:00.625260 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659157c-c527-43d0-a69c-54515e1dcac4 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:00.634720 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659157c-c527-43d0-a69c-54515e1dcac4 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:00.640808 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa4d40-c333-4ed5-922f-d6bd9ebc27ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:00.641565 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa4d40-c333-4ed5-922f-d6bd9ebc27ee admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:00.641923 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa4d40-c333-4ed5-922f-d6bd9ebc27ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:00.642290 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa4d40-c333-4ed5-922f-d6bd9ebc27ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:00.643005 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aa4d40-c333-4ed5-922f-d6bd9ebc27ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:00.645047 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aa4d40-c333-4ed5-922f-d6bd9ebc27ee admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:00.741598 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1312/2592] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:27:00 2022] POST /identity/v3/projects => generated 363 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:27:00.746570 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa3070-4644-45e2-94ad-ca8042a10de9 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:00.757107 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa3070-4644-45e2-94ad-ca8042a10de9 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:00.816048 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf13d5-0e26-43d6-87bc-96f62a86abec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:00.816669 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf13d5-0e26-43d6-87bc-96f62a86abec admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:00.816954 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf13d5-0e26-43d6-87bc-96f62a86abec admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:00.817233 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf13d5-0e26-43d6-87bc-96f62a86abec admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:00.817861 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf13d5-0e26-43d6-87bc-96f62a86abec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:00.819857 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf13d5-0e26-43d6-87bc-96f62a86abec admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:00.909425 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1281/2593] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:27:00 2022] POST /identity/v3/users => generated 282 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:27:00.913648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa4d40-c333-4ed5-922f-d6bd9ebc27ee admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:00.924502 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa4d40-c333-4ed5-922f-d6bd9ebc27ee admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:00.929092 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3765d8ce-9c5b-4eeb-887d-08537e565f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:00.929692 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765d8ce-9c5b-4eeb-887d-08537e565f5c admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:00.929974 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765d8ce-9c5b-4eeb-887d-08537e565f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:00.930252 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765d8ce-9c5b-4eeb-887d-08537e565f5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:00.930895 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3765d8ce-9c5b-4eeb-887d-08537e565f5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:00.932537 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3765d8ce-9c5b-4eeb-887d-08537e565f5c admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:00.943257 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1313/2594] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:27:00.947214 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf13d5-0e26-43d6-87bc-96f62a86abec admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:00.957711 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf13d5-0e26-43d6-87bc-96f62a86abec admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:00.962270 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b901d7e-a381-49bd-aea2-53f08ad89a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:00.966194 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b901d7e-a381-49bd-aea2-53f08ad89a49 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:00.966194 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b901d7e-a381-49bd-aea2-53f08ad89a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:00.966194 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b901d7e-a381-49bd-aea2-53f08ad89a49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:00.966194 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b901d7e-a381-49bd-aea2-53f08ad89a49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:00.966194 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b901d7e-a381-49bd-aea2-53f08ad89a49 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:00.971332 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:27:00.977126 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1282/2595] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:27:00.982149 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3765d8ce-9c5b-4eeb-887d-08537e565f5c admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:00.992364 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3765d8ce-9c5b-4eeb-887d-08537e565f5c admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:00.997762 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf99f325-e2b4-4b7a-854d-3ba69ffb5a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:00.998436 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99f325-e2b4-4b7a-854d-3ba69ffb5a12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:00.998716 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99f325-e2b4-4b7a-854d-3ba69ffb5a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:00.998999 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99f325-e2b4-4b7a-854d-3ba69ffb5a12 admin admin] PATH_INFO: `/v3/projects/eed85daa789d426d80195fd64839c279/users/9921f8767a8c4ba69114d62d6491ae85/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:01.014092 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf99f325-e2b4-4b7a-854d-3ba69ffb5a12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eed85daa789d426d80195fd64839c279, user_id=9921f8767a8c4ba69114d62d6491ae85, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:01.016785 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf99f325-e2b4-4b7a-854d-3ba69ffb5a12 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:01.089172 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1314/2596] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:27:00 2022] PUT /identity/v3/projects/eed85daa789d426d80195fd64839c279/users/9921f8767a8c4ba69114d62d6491ae85/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:27:01.101001 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe7e88-804f-41af-9fdc-021d0f727877 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:01.101391 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe7e88-804f-41af-9fdc-021d0f727877 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:01.101566 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe7e88-804f-41af-9fdc-021d0f727877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:01.130285 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-b7fe7e88-804f-41af-9fdc-021d0f727877 None None] MFA Rules not processed for user `9921f8767a8c4ba69114d62d6491ae85`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:01.131927 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-b7fe7e88-804f-41af-9fdc-021d0f727877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:01.202121 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1283/2597] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:27:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:27:01.207268 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf99f325-e2b4-4b7a-854d-3ba69ffb5a12 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:01.270866 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf99f325-e2b4-4b7a-854d-3ba69ffb5a12 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:01.275723 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e228b1-0f99-46a6-a4bb-5697542e9cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:01.278081 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e228b1-0f99-46a6-a4bb-5697542e9cdd admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:01.278393 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e228b1-0f99-46a6-a4bb-5697542e9cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:01.278674 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e228b1-0f99-46a6-a4bb-5697542e9cdd admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:01.279357 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e228b1-0f99-46a6-a4bb-5697542e9cdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:01.281321 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e228b1-0f99-46a6-a4bb-5697542e9cdd admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:01.376040 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1315/2598] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:27:01 2022] POST /identity/v3/projects => generated 361 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:27:01.380321 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe7e88-804f-41af-9fdc-021d0f727877 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:01.391509 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe7e88-804f-41af-9fdc-021d0f727877 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:01.447726 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b302899-2ed5-4498-9cf8-ff80445d94d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:01.448644 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b302899-2ed5-4498-9cf8-ff80445d94d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:01.452221 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b302899-2ed5-4498-9cf8-ff80445d94d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:01.452221 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b302899-2ed5-4498-9cf8-ff80445d94d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:01.452221 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b302899-2ed5-4498-9cf8-ff80445d94d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:01.454232 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b302899-2ed5-4498-9cf8-ff80445d94d4 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:01.563279 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1284/2599] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:27:01 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:27:01.567453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e228b1-0f99-46a6-a4bb-5697542e9cdd admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:01.578578 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e228b1-0f99-46a6-a4bb-5697542e9cdd admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:01.583182 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050eeca-e2a1-41e1-a791-65e27a8b0c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:01.584296 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050eeca-e2a1-41e1-a791-65e27a8b0c7f admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:01.584296 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050eeca-e2a1-41e1-a791-65e27a8b0c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:01.584511 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050eeca-e2a1-41e1-a791-65e27a8b0c7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:01.584965 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e050eeca-e2a1-41e1-a791-65e27a8b0c7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:01.586569 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e050eeca-e2a1-41e1-a791-65e27a8b0c7f admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:01.598721 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1316/2600] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:27:01.603011 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b302899-2ed5-4498-9cf8-ff80445d94d4 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:01.613349 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b302899-2ed5-4498-9cf8-ff80445d94d4 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:01.618019 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf9668-9a81-4ca1-9a42-ee90bf244e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:01.618678 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf9668-9a81-4ca1-9a42-ee90bf244e14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:01.618957 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf9668-9a81-4ca1-9a42-ee90bf244e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:01.619274 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf9668-9a81-4ca1-9a42-ee90bf244e14 admin admin] PATH_INFO: `/v3/projects/cec5fab8dbcf4c158692ca520a74d23a/users/0771435c76ae489b95f77ba2bc0ecc0c/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:01.633834 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbf9668-9a81-4ca1-9a42-ee90bf244e14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cec5fab8dbcf4c158692ca520a74d23a, user_id=0771435c76ae489b95f77ba2bc0ecc0c, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:01.637065 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbf9668-9a81-4ca1-9a42-ee90bf244e14 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:01.682571 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1285/2601] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:27:01 2022] PUT /identity/v3/projects/cec5fab8dbcf4c158692ca520a74d23a/users/0771435c76ae489b95f77ba2bc0ecc0c/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:27:01.686473 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050eeca-e2a1-41e1-a791-65e27a8b0c7f admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:01.697411 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050eeca-e2a1-41e1-a791-65e27a8b0c7f admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:01.756491 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3934aa6-5516-4fe1-a2f6-51471dbffb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:01.757117 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3934aa6-5516-4fe1-a2f6-51471dbffb71 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:01.757404 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3934aa6-5516-4fe1-a2f6-51471dbffb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:01.757681 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3934aa6-5516-4fe1-a2f6-51471dbffb71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:01.758334 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3934aa6-5516-4fe1-a2f6-51471dbffb71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:01.760034 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3934aa6-5516-4fe1-a2f6-51471dbffb71 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:01.770935 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1317/2602] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:27:01.775128 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf9668-9a81-4ca1-9a42-ee90bf244e14 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:01.786315 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf9668-9a81-4ca1-9a42-ee90bf244e14 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:01.791180 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca30e596-a47d-44f5-aeb3-4e16cfe8cf7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:01.791976 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca30e596-a47d-44f5-aeb3-4e16cfe8cf7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:01.792339 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca30e596-a47d-44f5-aeb3-4e16cfe8cf7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:01.792714 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca30e596-a47d-44f5-aeb3-4e16cfe8cf7f admin admin] PATH_INFO: `/v3/projects/cec5fab8dbcf4c158692ca520a74d23a/users/0771435c76ae489b95f77ba2bc0ecc0c/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:01.794725 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca30e596-a47d-44f5-aeb3-4e16cfe8cf7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cec5fab8dbcf4c158692ca520a74d23a, user_id=0771435c76ae489b95f77ba2bc0ecc0c, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:01.797480 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca30e596-a47d-44f5-aeb3-4e16cfe8cf7f admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:01.808301 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1286/2603] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:27:01 2022] PUT /identity/v3/projects/cec5fab8dbcf4c158692ca520a74d23a/users/0771435c76ae489b95f77ba2bc0ecc0c/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:27:01.818710 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb0e93-d9c7-4715-9cab-8fda8949cc4c None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:01.819009 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb0e93-d9c7-4715-9cab-8fda8949cc4c None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:01.819338 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb0e93-d9c7-4715-9cab-8fda8949cc4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:01.827708 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:27:01.850510 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-c4cb0e93-d9c7-4715-9cab-8fda8949cc4c None None] MFA Rules not processed for user `0771435c76ae489b95f77ba2bc0ecc0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:01.859586 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-c4cb0e93-d9c7-4715-9cab-8fda8949cc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:01.925182 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1318/2604] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:27:01 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:27:01.936927 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca30e596-a47d-44f5-aeb3-4e16cfe8cf7f admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:01.938671 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-ca30e596-a47d-44f5-aeb3-4e16cfe8cf7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:01.985351 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:27:01.997363 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca30e596-a47d-44f5-aeb3-4e16cfe8cf7f admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:02.001092 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa8494-d1bd-4b61-bcbc-ced2018656d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:02.001697 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa8494-d1bd-4b61-bcbc-ced2018656d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:02.001983 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa8494-d1bd-4b61-bcbc-ced2018656d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:02.002259 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa8494-d1bd-4b61-bcbc-ced2018656d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:02.012241 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa8494-d1bd-4b61-bcbc-ced2018656d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:02.013668 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa8494-d1bd-4b61-bcbc-ced2018656d8 service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:02.023117 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1287/2605] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:27:01 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:27:02.214463 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb0e93-d9c7-4715-9cab-8fda8949cc4c None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:02.224479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb0e93-d9c7-4715-9cab-8fda8949cc4c None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:02.228000 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab9bc0-9025-463f-8e1f-95185b271080 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:02.228605 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab9bc0-9025-463f-8e1f-95185b271080 service cinder] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:02.228884 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab9bc0-9025-463f-8e1f-95185b271080 service cinder] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:02.229158 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab9bc0-9025-463f-8e1f-95185b271080 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:02.290783 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab9bc0-9025-463f-8e1f-95185b271080 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:02.292367 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab9bc0-9025-463f-8e1f-95185b271080 service cinder] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:02.302594 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1319/2606] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:27:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:27:05.861151 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa8494-d1bd-4b61-bcbc-ced2018656d8 service cinder] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:05.923052 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa8494-d1bd-4b61-bcbc-ced2018656d8 service cinder] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:05.927590 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b15878-f28f-483c-a595-13af3dc9e50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:05.928259 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b15878-f28f-483c-a595-13af3dc9e50a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:05.928543 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b15878-f28f-483c-a595-13af3dc9e50a admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:05.928829 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b15878-f28f-483c-a595-13af3dc9e50a admin admin] PATH_INFO: `/v3/users/9921f8767a8c4ba69114d62d6491ae85` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:05.929958 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b15878-f28f-483c-a595-13af3dc9e50a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9921f8767a8c4ba69114d62d6491ae85)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:05.931912 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b15878-f28f-483c-a595-13af3dc9e50a admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:06.106528 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-d8b15878-f28f-483c-a595-13af3dc9e50a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9921f8767a8c4ba69114d62d6491ae85'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:06.107132 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-d8b15878-f28f-483c-a595-13af3dc9e50a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9921f8767a8c4ba69114d62d6491ae85'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:06.107752 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-d8b15878-f28f-483c-a595-13af3dc9e50a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9921f8767a8c4ba69114d62d6491ae85'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:06.114937 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-d8b15878-f28f-483c-a595-13af3dc9e50a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9921f8767a8c4ba69114d62d6491ae85'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:06.127919 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-d8b15878-f28f-483c-a595-13af3dc9e50a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9921f8767a8c4ba69114d62d6491ae85'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:06.206094 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1288/2607] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:27:05 2022] DELETE /identity/v3/users/9921f8767a8c4ba69114d62d6491ae85 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:27:06.210588 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05a67a-0e4b-4f7e-b18b-247d70fae83e None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:06.211090 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05a67a-0e4b-4f7e-b18b-247d70fae83e None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:06.211199 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05a67a-0e4b-4f7e-b18b-247d70fae83e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:06.230694 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-0b05a67a-0e4b-4f7e-b18b-247d70fae83e None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:06.231932 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-0b05a67a-0e4b-4f7e-b18b-247d70fae83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:06.290566 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1320/2608] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:27:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:27:06.301329 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b15878-f28f-483c-a595-13af3dc9e50a admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:06.302758 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-d8b15878-f28f-483c-a595-13af3dc9e50a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:06.361050 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b15878-f28f-483c-a595-13af3dc9e50a admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:06.364783 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88737e89-c4c3-4420-97d3-dc5980b06e1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:06.365448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88737e89-c4c3-4420-97d3-dc5980b06e1a service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:06.365842 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88737e89-c4c3-4420-97d3-dc5980b06e1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:06.366017 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88737e89-c4c3-4420-97d3-dc5980b06e1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:06.377111 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88737e89-c4c3-4420-97d3-dc5980b06e1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:06.378563 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88737e89-c4c3-4420-97d3-dc5980b06e1a service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:06.392026 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1289/2609] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:27:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:27:06.836181 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b05a67a-0e4b-4f7e-b18b-247d70fae83e None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:06.837689 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-0b05a67a-0e4b-4f7e-b18b-247d70fae83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:06.853481 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b05a67a-0e4b-4f7e-b18b-247d70fae83e None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:06.859614 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab860e59-f9e8-488f-93d9-ec28d0b18bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:06.860219 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab860e59-f9e8-488f-93d9-ec28d0b18bb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:06.860506 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab860e59-f9e8-488f-93d9-ec28d0b18bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:06.860801 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab860e59-f9e8-488f-93d9-ec28d0b18bb6 admin admin] PATH_INFO: `/v3/projects/eed85daa789d426d80195fd64839c279` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:06.861925 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab860e59-f9e8-488f-93d9-ec28d0b18bb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eed85daa789d426d80195fd64839c279)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:06.863887 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab860e59-f9e8-488f-93d9-ec28d0b18bb6 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:06.887641 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:27:07.003556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-ab860e59-f9e8-488f-93d9-ec28d0b18bb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eed85daa789d426d80195fd64839c279', 'request_id': 'req-ab860e59-f9e8-488f-93d9-ec28d0b18bb6'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:07.009339 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-ab860e59-f9e8-488f-93d9-ec28d0b18bb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eed85daa789d426d80195fd64839c279', 'request_id': 'req-ab860e59-f9e8-488f-93d9-ec28d0b18bb6'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:07.076494 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-ab860e59-f9e8-488f-93d9-ec28d0b18bb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eed85daa789d426d80195fd64839c279', 'request_id': 'req-ab860e59-f9e8-488f-93d9-ec28d0b18bb6'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:07.088403 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-ab860e59-f9e8-488f-93d9-ec28d0b18bb6 admin admin] The token cache is being invalidate because project eed85daa789d426d80195fd64839c279 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:27:07.088698 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-ab860e59-f9e8-488f-93d9-ec28d0b18bb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:07.089983 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-ab860e59-f9e8-488f-93d9-ec28d0b18bb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:07.091610 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1321/2610] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:27:06 2022] DELETE /identity/v3/projects/eed85daa789d426d80195fd64839c279 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:27:07.095657 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88737e89-c4c3-4420-97d3-dc5980b06e1a service neutron] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:07.105213 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88737e89-c4c3-4420-97d3-dc5980b06e1a service neutron] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:07.109479 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:27:07.162018 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc400b02-5d4a-442a-809d-b3937d096e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:07.162761 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc400b02-5d4a-442a-809d-b3937d096e80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:07.163100 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc400b02-5d4a-442a-809d-b3937d096e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:07.163653 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc400b02-5d4a-442a-809d-b3937d096e80 admin admin] PATH_INFO: `/v3/users/0771435c76ae489b95f77ba2bc0ecc0c` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:07.165281 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc400b02-5d4a-442a-809d-b3937d096e80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0771435c76ae489b95f77ba2bc0ecc0c)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:07.167293 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc400b02-5d4a-442a-809d-b3937d096e80 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:07.391992 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-bc400b02-5d4a-442a-809d-b3937d096e80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0771435c76ae489b95f77ba2bc0ecc0c'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:07.392659 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-bc400b02-5d4a-442a-809d-b3937d096e80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0771435c76ae489b95f77ba2bc0ecc0c'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:07.393272 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-bc400b02-5d4a-442a-809d-b3937d096e80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0771435c76ae489b95f77ba2bc0ecc0c'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:07.401400 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-bc400b02-5d4a-442a-809d-b3937d096e80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0771435c76ae489b95f77ba2bc0ecc0c'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:07.416550 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-bc400b02-5d4a-442a-809d-b3937d096e80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0771435c76ae489b95f77ba2bc0ecc0c'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:07.515229 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1290/2611] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:27:07 2022] DELETE /identity/v3/users/0771435c76ae489b95f77ba2bc0ecc0c => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:27:07.949636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab860e59-f9e8-488f-93d9-ec28d0b18bb6 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:07.951513 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-ab860e59-f9e8-488f-93d9-ec28d0b18bb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:07.968696 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab860e59-f9e8-488f-93d9-ec28d0b18bb6 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:08.029142 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe1e82b-3188-4d50-9750-501c111d9321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:08.029781 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe1e82b-3188-4d50-9750-501c111d9321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:08.030066 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe1e82b-3188-4d50-9750-501c111d9321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:08.030344 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe1e82b-3188-4d50-9750-501c111d9321 admin admin] PATH_INFO: `/v3/projects/cec5fab8dbcf4c158692ca520a74d23a` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:08.031519 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe1e82b-3188-4d50-9750-501c111d9321 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cec5fab8dbcf4c158692ca520a74d23a)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:08.033456 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe1e82b-3188-4d50-9750-501c111d9321 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:08.194900 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-abe1e82b-3188-4d50-9750-501c111d9321 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cec5fab8dbcf4c158692ca520a74d23a', 'request_id': 'req-abe1e82b-3188-4d50-9750-501c111d9321'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:08.197830 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-abe1e82b-3188-4d50-9750-501c111d9321 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cec5fab8dbcf4c158692ca520a74d23a', 'request_id': 'req-abe1e82b-3188-4d50-9750-501c111d9321'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:08.294179 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-abe1e82b-3188-4d50-9750-501c111d9321 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cec5fab8dbcf4c158692ca520a74d23a', 'request_id': 'req-abe1e82b-3188-4d50-9750-501c111d9321'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:08.308640 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-abe1e82b-3188-4d50-9750-501c111d9321 admin admin] The token cache is being invalidate because project cec5fab8dbcf4c158692ca520a74d23a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:27:08.308942 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-abe1e82b-3188-4d50-9750-501c111d9321 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:08.309815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-abe1e82b-3188-4d50-9750-501c111d9321 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:08.315065 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1322/2612] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:27:07 2022] DELETE /identity/v3/projects/cec5fab8dbcf4c158692ca520a74d23a => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:27:08.316716 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a4348-e79b-45d1-a6ba-7a07a8210e12 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:08.317199 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a4348-e79b-45d1-a6ba-7a07a8210e12 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:08.317300 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a4348-e79b-45d1-a6ba-7a07a8210e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:08.337361 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-cd4a4348-e79b-45d1-a6ba-7a07a8210e12 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:08.338593 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-cd4a4348-e79b-45d1-a6ba-7a07a8210e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:08.403374 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1291/2613] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:27:08.410049 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beff065-5583-49a2-9b1d-f3f80b5da15f None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:08.410346 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beff065-5583-49a2-9b1d-f3f80b5da15f None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:08.410654 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beff065-5583-49a2-9b1d-f3f80b5da15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:08.431991 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-8beff065-5583-49a2-9b1d-f3f80b5da15f None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:08.438220 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-8beff065-5583-49a2-9b1d-f3f80b5da15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:08.441734 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1323/2614] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:27:08.446098 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a4348-e79b-45d1-a6ba-7a07a8210e12 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:08.456550 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a4348-e79b-45d1-a6ba-7a07a8210e12 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:08.461215 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa4076-835d-4def-9c73-ec828d89aaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:08.461882 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa4076-835d-4def-9c73-ec828d89aaa7 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:08.462115 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa4076-835d-4def-9c73-ec828d89aaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:08.462384 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa4076-835d-4def-9c73-ec828d89aaa7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:08.463035 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaa4076-835d-4def-9c73-ec828d89aaa7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:27:08.463335 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaa4076-835d-4def-9c73-ec828d89aaa7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:08.465774 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaa4076-835d-4def-9c73-ec828d89aaa7 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:08.477112 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1292/2615] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:27:08.481812 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beff065-5583-49a2-9b1d-f3f80b5da15f None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:08.491279 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beff065-5583-49a2-9b1d-f3f80b5da15f None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:08.496038 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dcd06a-3c41-406c-a2aa-8310895ef945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:08.500983 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcd06a-3c41-406c-a2aa-8310895ef945 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:08.500983 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcd06a-3c41-406c-a2aa-8310895ef945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:08.500983 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcd06a-3c41-406c-a2aa-8310895ef945 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:08.500983 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dcd06a-3c41-406c-a2aa-8310895ef945 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:08.500983 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dcd06a-3c41-406c-a2aa-8310895ef945 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:08.627220 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1324/2616] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:27:08 2022] POST /identity/v3/projects => generated 387 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:27:08.634868 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa4076-835d-4def-9c73-ec828d89aaa7 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:08.645703 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa4076-835d-4def-9c73-ec828d89aaa7 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:08.654919 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:27:08.710289 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c6529-6853-44c5-ab63-00b6e3b3fa4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:08.712561 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c6529-6853-44c5-ab63-00b6e3b3fa4e admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:08.712561 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c6529-6853-44c5-ab63-00b6e3b3fa4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:08.712561 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c6529-6853-44c5-ab63-00b6e3b3fa4e admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:08.712561 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c6529-6853-44c5-ab63-00b6e3b3fa4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:08.714324 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c6529-6853-44c5-ab63-00b6e3b3fa4e admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:08.806047 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1293/2617] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:27:08 2022] POST /identity/v3/users => generated 294 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:27:08.812830 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dcd06a-3c41-406c-a2aa-8310895ef945 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:08.822405 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dcd06a-3c41-406c-a2aa-8310895ef945 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:08.828584 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b3586-8455-4e1c-90b7-9c64a873733a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:08.828814 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b3586-8455-4e1c-90b7-9c64a873733a admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:08.829190 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b3586-8455-4e1c-90b7-9c64a873733a admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:08.829484 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b3586-8455-4e1c-90b7-9c64a873733a admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:08.830307 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b3586-8455-4e1c-90b7-9c64a873733a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:08.831967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b3586-8455-4e1c-90b7-9c64a873733a admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:08.837762 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:27:08.843341 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1325/2618] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:27:08.847550 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c6529-6853-44c5-ab63-00b6e3b3fa4e admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:08.857963 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c6529-6853-44c5-ab63-00b6e3b3fa4e admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:08.863287 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1c00e9-40ac-42f8-bc38-90732cbeb27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:08.864168 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c00e9-40ac-42f8-bc38-90732cbeb27a admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:08.864699 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c00e9-40ac-42f8-bc38-90732cbeb27a admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:08.864978 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c00e9-40ac-42f8-bc38-90732cbeb27a admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:08.865870 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1c00e9-40ac-42f8-bc38-90732cbeb27a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:08.868001 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1c00e9-40ac-42f8-bc38-90732cbeb27a admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:08.881413 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1294/2619] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:27:08.885047 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b3586-8455-4e1c-90b7-9c64a873733a admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:08.896825 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b3586-8455-4e1c-90b7-9c64a873733a admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:08.901799 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e1f7a-d0e6-4abd-ab97-e201fe0beddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:08.902791 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e1f7a-d0e6-4abd-ab97-e201fe0beddc admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:08.904652 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e1f7a-d0e6-4abd-ab97-e201fe0beddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:08.904652 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e1f7a-d0e6-4abd-ab97-e201fe0beddc admin admin] PATH_INFO: `/v3/projects/7a203c444ff94c1e94d4ce2261bd663e/users/b2a1f0bc07264603bcc55e97a2291d1e/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:08.920569 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e1f7a-d0e6-4abd-ab97-e201fe0beddc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a203c444ff94c1e94d4ce2261bd663e, user_id=b2a1f0bc07264603bcc55e97a2291d1e, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:08.923260 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e1f7a-d0e6-4abd-ab97-e201fe0beddc admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:09.041529 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1326/2620] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:27:08 2022] PUT /identity/v3/projects/7a203c444ff94c1e94d4ce2261bd663e/users/b2a1f0bc07264603bcc55e97a2291d1e/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:27:09.053567 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3058cb-06f1-4480-acd1-47f5a28af147 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:09.053851 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3058cb-06f1-4480-acd1-47f5a28af147 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:09.054133 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3058cb-06f1-4480-acd1-47f5a28af147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:09.088599 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-5d3058cb-06f1-4480-acd1-47f5a28af147 None None] MFA Rules not processed for user `b2a1f0bc07264603bcc55e97a2291d1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:09.089627 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-5d3058cb-06f1-4480-acd1-47f5a28af147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:09.158440 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1295/2621] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:27:09 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:27:09.162902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e1f7a-d0e6-4abd-ab97-e201fe0beddc admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:09.225922 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e1f7a-d0e6-4abd-ab97-e201fe0beddc admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:09.230869 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b14fd2-61c2-4881-b78c-a29e5fe03ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:09.231839 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b14fd2-61c2-4881-b78c-a29e5fe03ff9 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:09.232123 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b14fd2-61c2-4881-b78c-a29e5fe03ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:09.232397 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b14fd2-61c2-4881-b78c-a29e5fe03ff9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:09.233014 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b14fd2-61c2-4881-b78c-a29e5fe03ff9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:09.234925 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b14fd2-61c2-4881-b78c-a29e5fe03ff9 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:09.332916 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1327/2622] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:27:09 2022] POST /identity/v3/projects => generated 389 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:27:09.336967 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3058cb-06f1-4480-acd1-47f5a28af147 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:09.348590 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3058cb-06f1-4480-acd1-47f5a28af147 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:09.413869 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db36bb5-63ee-4b82-a09b-6443ccb9711d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:09.414493 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db36bb5-63ee-4b82-a09b-6443ccb9711d admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:09.414952 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db36bb5-63ee-4b82-a09b-6443ccb9711d admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:09.415123 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db36bb5-63ee-4b82-a09b-6443ccb9711d admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:09.417730 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db36bb5-63ee-4b82-a09b-6443ccb9711d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:09.419689 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db36bb5-63ee-4b82-a09b-6443ccb9711d admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:09.494922 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1296/2623] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:27:09 2022] POST /identity/v3/users => generated 301 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:27:09.496658 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b14fd2-61c2-4881-b78c-a29e5fe03ff9 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:09.507119 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b14fd2-61c2-4881-b78c-a29e5fe03ff9 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:09.512550 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef2abe2-71d3-4e48-a013-545c0b339f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:09.513157 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef2abe2-71d3-4e48-a013-545c0b339f23 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:09.513442 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef2abe2-71d3-4e48-a013-545c0b339f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:09.513723 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef2abe2-71d3-4e48-a013-545c0b339f23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:09.514365 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef2abe2-71d3-4e48-a013-545c0b339f23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:09.515986 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef2abe2-71d3-4e48-a013-545c0b339f23 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:09.526948 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1328/2624] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:27:09.531190 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db36bb5-63ee-4b82-a09b-6443ccb9711d admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:09.541233 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db36bb5-63ee-4b82-a09b-6443ccb9711d admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:09.545913 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe49d75d-87b3-4b3f-a5d8-8c893bf1f958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:09.546558 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49d75d-87b3-4b3f-a5d8-8c893bf1f958 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:09.546843 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49d75d-87b3-4b3f-a5d8-8c893bf1f958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:09.547119 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49d75d-87b3-4b3f-a5d8-8c893bf1f958 admin admin] PATH_INFO: `/v3/projects/3774701788374f28b460244f98d683dc/users/9035212f9bb64a76b8ecdd0051e077b2/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:09.555954 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:27:09.563129 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe49d75d-87b3-4b3f-a5d8-8c893bf1f958 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3774701788374f28b460244f98d683dc, user_id=9035212f9bb64a76b8ecdd0051e077b2, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:09.565818 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe49d75d-87b3-4b3f-a5d8-8c893bf1f958 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:09.662938 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1297/2625] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:27:09 2022] PUT /identity/v3/projects/3774701788374f28b460244f98d683dc/users/9035212f9bb64a76b8ecdd0051e077b2/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:27:09.667147 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef2abe2-71d3-4e48-a013-545c0b339f23 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:09.677267 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef2abe2-71d3-4e48-a013-545c0b339f23 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:09.739009 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ebe37d-d304-4bee-b917-a67c92dc6214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:09.739552 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ebe37d-d304-4bee-b917-a67c92dc6214 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:09.739840 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ebe37d-d304-4bee-b917-a67c92dc6214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:09.740118 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ebe37d-d304-4bee-b917-a67c92dc6214 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:09.740769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ebe37d-d304-4bee-b917-a67c92dc6214 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:09.742380 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ebe37d-d304-4bee-b917-a67c92dc6214 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:09.753345 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1329/2626] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:27:09.761534 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe49d75d-87b3-4b3f-a5d8-8c893bf1f958 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:09.769165 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe49d75d-87b3-4b3f-a5d8-8c893bf1f958 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:09.774067 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e50895-3cd2-4db6-88bd-bda96d1d7130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:09.774729 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e50895-3cd2-4db6-88bd-bda96d1d7130 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:09.775013 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e50895-3cd2-4db6-88bd-bda96d1d7130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:09.775291 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e50895-3cd2-4db6-88bd-bda96d1d7130 admin admin] PATH_INFO: `/v3/projects/3774701788374f28b460244f98d683dc/users/9035212f9bb64a76b8ecdd0051e077b2/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:09.777291 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e50895-3cd2-4db6-88bd-bda96d1d7130 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3774701788374f28b460244f98d683dc, user_id=9035212f9bb64a76b8ecdd0051e077b2, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:09.779960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e50895-3cd2-4db6-88bd-bda96d1d7130 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:09.790507 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1298/2627] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:27:09 2022] PUT /identity/v3/projects/3774701788374f28b460244f98d683dc/users/9035212f9bb64a76b8ecdd0051e077b2/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:27:09.801355 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26bff7-4feb-44bf-b93f-f38f151288e0 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:09.804633 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26bff7-4feb-44bf-b93f-f38f151288e0 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:09.804633 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26bff7-4feb-44bf-b93f-f38f151288e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:09.830260 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-bf26bff7-4feb-44bf-b93f-f38f151288e0 None None] MFA Rules not processed for user `9035212f9bb64a76b8ecdd0051e077b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:09.831525 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-bf26bff7-4feb-44bf-b93f-f38f151288e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:09.869127 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:27:09.906749 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1330/2628] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:27:09 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:27:09.917946 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e50895-3cd2-4db6-88bd-bda96d1d7130 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:09.919707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-e6e50895-3cd2-4db6-88bd-bda96d1d7130 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:09.981934 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e50895-3cd2-4db6-88bd-bda96d1d7130 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:09.985708 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57756033-ab3c-4ae2-a4e8-3f2dda3d47bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:09.986330 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57756033-ab3c-4ae2-a4e8-3f2dda3d47bc service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:09.986669 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57756033-ab3c-4ae2-a4e8-3f2dda3d47bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:09.986949 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57756033-ab3c-4ae2-a4e8-3f2dda3d47bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:09.997070 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57756033-ab3c-4ae2-a4e8-3f2dda3d47bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:09.998496 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57756033-ab3c-4ae2-a4e8-3f2dda3d47bc service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:10.010602 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1299/2629] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:27:09 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:27:10.168185 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf26bff7-4feb-44bf-b93f-f38f151288e0 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:10.232415 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf26bff7-4feb-44bf-b93f-f38f151288e0 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:10.236966 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d11ed89-0a92-455d-acd5-eef0bcc69ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:10.237592 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d11ed89-0a92-455d-acd5-eef0bcc69ac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:10.237872 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d11ed89-0a92-455d-acd5-eef0bcc69ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:10.238147 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d11ed89-0a92-455d-acd5-eef0bcc69ac9 admin admin] PATH_INFO: `/v3/users/b2a1f0bc07264603bcc55e97a2291d1e` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:10.239265 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d11ed89-0a92-455d-acd5-eef0bcc69ac9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2a1f0bc07264603bcc55e97a2291d1e)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:10.241291 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d11ed89-0a92-455d-acd5-eef0bcc69ac9 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:10.456402 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-6d11ed89-0a92-455d-acd5-eef0bcc69ac9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2a1f0bc07264603bcc55e97a2291d1e'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:10.463258 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-6d11ed89-0a92-455d-acd5-eef0bcc69ac9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2a1f0bc07264603bcc55e97a2291d1e'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:10.464500 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-6d11ed89-0a92-455d-acd5-eef0bcc69ac9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2a1f0bc07264603bcc55e97a2291d1e'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:10.477119 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-6d11ed89-0a92-455d-acd5-eef0bcc69ac9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2a1f0bc07264603bcc55e97a2291d1e'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:10.562422 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-6d11ed89-0a92-455d-acd5-eef0bcc69ac9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2a1f0bc07264603bcc55e97a2291d1e'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:10.568878 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1331/2630] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:27:10 2022] DELETE /identity/v3/users/b2a1f0bc07264603bcc55e97a2291d1e => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:27:10.574012 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726cbfc-4726-4c22-a580-28f7c45e022e None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:10.574502 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726cbfc-4726-4c22-a580-28f7c45e022e None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:10.574626 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726cbfc-4726-4c22-a580-28f7c45e022e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:10.595198 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-3726cbfc-4726-4c22-a580-28f7c45e022e None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:10.596469 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-3726cbfc-4726-4c22-a580-28f7c45e022e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:10.656642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1300/2631] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:27:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:27:10.675433 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d11ed89-0a92-455d-acd5-eef0bcc69ac9 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:10.676912 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-6d11ed89-0a92-455d-acd5-eef0bcc69ac9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:10.737368 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d11ed89-0a92-455d-acd5-eef0bcc69ac9 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:10.741286 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf75d5c-764d-4f40-8ba1-3985b13ee298 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:10.747340 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf75d5c-764d-4f40-8ba1-3985b13ee298 service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:10.747340 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf75d5c-764d-4f40-8ba1-3985b13ee298 service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:10.747340 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf75d5c-764d-4f40-8ba1-3985b13ee298 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:10.748476 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:27:10.752306 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf75d5c-764d-4f40-8ba1-3985b13ee298 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:10.753738 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf75d5c-764d-4f40-8ba1-3985b13ee298 service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:10.763943 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1332/2632] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:27:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:27:11.496256 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3726cbfc-4726-4c22-a580-28f7c45e022e None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:11.497692 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-3726cbfc-4726-4c22-a580-28f7c45e022e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:11.509575 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3726cbfc-4726-4c22-a580-28f7c45e022e None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:11.516958 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a2e055-0db2-4587-b7ca-755b4223227d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:11.516958 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a2e055-0db2-4587-b7ca-755b4223227d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:11.516958 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a2e055-0db2-4587-b7ca-755b4223227d admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:11.516958 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a2e055-0db2-4587-b7ca-755b4223227d admin admin] PATH_INFO: `/v3/projects/7a203c444ff94c1e94d4ce2261bd663e` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:11.517394 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a2e055-0db2-4587-b7ca-755b4223227d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a203c444ff94c1e94d4ce2261bd663e)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:11.518510 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a2e055-0db2-4587-b7ca-755b4223227d admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:11.787103 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-02a2e055-0db2-4587-b7ca-755b4223227d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a203c444ff94c1e94d4ce2261bd663e', 'request_id': 'req-02a2e055-0db2-4587-b7ca-755b4223227d'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:11.793293 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-02a2e055-0db2-4587-b7ca-755b4223227d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a203c444ff94c1e94d4ce2261bd663e', 'request_id': 'req-02a2e055-0db2-4587-b7ca-755b4223227d'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:11.857226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-02a2e055-0db2-4587-b7ca-755b4223227d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a203c444ff94c1e94d4ce2261bd663e', 'request_id': 'req-02a2e055-0db2-4587-b7ca-755b4223227d'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:11.866758 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:27:11.870810 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-02a2e055-0db2-4587-b7ca-755b4223227d admin admin] The token cache is being invalidate because project 7a203c444ff94c1e94d4ce2261bd663e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:27:11.871102 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-02a2e055-0db2-4587-b7ca-755b4223227d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:11.871891 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-02a2e055-0db2-4587-b7ca-755b4223227d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:11.873413 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1301/2633] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:27:11 2022] DELETE /identity/v3/projects/7a203c444ff94c1e94d4ce2261bd663e => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:27:11.877516 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf75d5c-764d-4f40-8ba1-3985b13ee298 service neutron] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:11.887792 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf75d5c-764d-4f40-8ba1-3985b13ee298 service neutron] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:11.948382 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff75677-e841-4f87-bae9-4227ca300cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:11.949020 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff75677-e841-4f87-bae9-4227ca300cc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:11.949304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff75677-e841-4f87-bae9-4227ca300cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:11.949581 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff75677-e841-4f87-bae9-4227ca300cc9 admin admin] PATH_INFO: `/v3/users/9035212f9bb64a76b8ecdd0051e077b2` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:11.950711 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff75677-e841-4f87-bae9-4227ca300cc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9035212f9bb64a76b8ecdd0051e077b2)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:11.952659 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff75677-e841-4f87-bae9-4227ca300cc9 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:12.113504 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-aff75677-e841-4f87-bae9-4227ca300cc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9035212f9bb64a76b8ecdd0051e077b2'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:12.120273 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-aff75677-e841-4f87-bae9-4227ca300cc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9035212f9bb64a76b8ecdd0051e077b2'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:12.120939 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-aff75677-e841-4f87-bae9-4227ca300cc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9035212f9bb64a76b8ecdd0051e077b2'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:12.133453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-aff75677-e841-4f87-bae9-4227ca300cc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9035212f9bb64a76b8ecdd0051e077b2'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:12.202588 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-aff75677-e841-4f87-bae9-4227ca300cc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9035212f9bb64a76b8ecdd0051e077b2'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:12.209117 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1333/2634] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:27:11 2022] DELETE /identity/v3/users/9035212f9bb64a76b8ecdd0051e077b2 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:27:12.397491 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a2e055-0db2-4587-b7ca-755b4223227d admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:12.397491 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-02a2e055-0db2-4587-b7ca-755b4223227d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:12.410143 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a2e055-0db2-4587-b7ca-755b4223227d admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:12.432452 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff75677-e841-4f87-bae9-4227ca300cc9 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:12.436396 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-aff75677-e841-4f87-bae9-4227ca300cc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:12.488814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59dbef-3664-4291-85ec-f549992ac3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:12.489649 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59dbef-3664-4291-85ec-f549992ac3e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:12.489995 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59dbef-3664-4291-85ec-f549992ac3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:12.490362 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59dbef-3664-4291-85ec-f549992ac3e4 admin admin] PATH_INFO: `/v3/users/a24d95a983f641288a4883ffcfb931cc` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:12.491607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce59dbef-3664-4291-85ec-f549992ac3e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a24d95a983f641288a4883ffcfb931cc)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:12.493918 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce59dbef-3664-4291-85ec-f549992ac3e4 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:12.522482 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff75677-e841-4f87-bae9-4227ca300cc9 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:12.528363 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae1c2a-05ce-4ea3-9eed-323c48719992 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:12.528995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae1c2a-05ce-4ea3-9eed-323c48719992 service nova] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:12.529277 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae1c2a-05ce-4ea3-9eed-323c48719992 service nova] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:12.529813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae1c2a-05ce-4ea3-9eed-323c48719992 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:12.531661 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-25ae1c2a-05ce-4ea3-9eed-323c48719992 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:12.541516 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:27:12.593396 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae1c2a-05ce-4ea3-9eed-323c48719992 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:12.595032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae1c2a-05ce-4ea3-9eed-323c48719992 service nova] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:12.604246 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1334/2635] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:27:12 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:27:12.689867 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae1c2a-05ce-4ea3-9eed-323c48719992 service nova] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:12.692747 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-25ae1c2a-05ce-4ea3-9eed-323c48719992 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:12.713361 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae1c2a-05ce-4ea3-9eed-323c48719992 service nova] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:12.721931 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3da641-b25a-4dfe-803d-c25ce358fc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:12.722621 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3da641-b25a-4dfe-803d-c25ce358fc42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:12.723051 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-ce59dbef-3664-4291-85ec-f549992ac3e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a24d95a983f641288a4883ffcfb931cc'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:12.723566 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3da641-b25a-4dfe-803d-c25ce358fc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:12.723905 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-ce59dbef-3664-4291-85ec-f549992ac3e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a24d95a983f641288a4883ffcfb931cc'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:12.724027 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3da641-b25a-4dfe-803d-c25ce358fc42 admin admin] PATH_INFO: `/v3/projects/3774701788374f28b460244f98d683dc` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:12.724516 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-ce59dbef-3664-4291-85ec-f549992ac3e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a24d95a983f641288a4883ffcfb931cc'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:12.725149 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3da641-b25a-4dfe-803d-c25ce358fc42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3774701788374f28b460244f98d683dc)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:12.727293 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3da641-b25a-4dfe-803d-c25ce358fc42 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:12.738884 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-ce59dbef-3664-4291-85ec-f549992ac3e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a24d95a983f641288a4883ffcfb931cc'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:12.753839 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-ce59dbef-3664-4291-85ec-f549992ac3e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a24d95a983f641288a4883ffcfb931cc'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:12.823207 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1302/2636] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:27:12 2022] DELETE /identity/v3/users/a24d95a983f641288a4883ffcfb931cc => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:27:12.827456 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300a7ef-7dc5-4d26-b9b9-00fe7fdb51d8 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:12.827745 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300a7ef-7dc5-4d26-b9b9-00fe7fdb51d8 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:12.828020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300a7ef-7dc5-4d26-b9b9-00fe7fdb51d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:12.846443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-8300a7ef-7dc5-4d26-b9b9-00fe7fdb51d8 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:12.847691 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-8300a7ef-7dc5-4d26-b9b9-00fe7fdb51d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:12.880991 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-8a3da641-b25a-4dfe-803d-c25ce358fc42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3774701788374f28b460244f98d683dc', 'request_id': 'req-8a3da641-b25a-4dfe-803d-c25ce358fc42'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:12.886451 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-8a3da641-b25a-4dfe-803d-c25ce358fc42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3774701788374f28b460244f98d683dc', 'request_id': 'req-8a3da641-b25a-4dfe-803d-c25ce358fc42'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:12.908120 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1303/2637] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:27:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:27:12.918307 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300a7ef-7dc5-4d26-b9b9-00fe7fdb51d8 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:12.919696 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-8300a7ef-7dc5-4d26-b9b9-00fe7fdb51d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:12.941373 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:27:12.975533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300a7ef-7dc5-4d26-b9b9-00fe7fdb51d8 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:12.990937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-8a3da641-b25a-4dfe-803d-c25ce358fc42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3774701788374f28b460244f98d683dc', 'request_id': 'req-8a3da641-b25a-4dfe-803d-c25ce358fc42'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:13.003780 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-8a3da641-b25a-4dfe-803d-c25ce358fc42 admin admin] The token cache is being invalidate because project 3774701788374f28b460244f98d683dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:27:13.004081 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-8a3da641-b25a-4dfe-803d-c25ce358fc42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:13.008131 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-8a3da641-b25a-4dfe-803d-c25ce358fc42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:13.008131 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1335/2638] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:27:12 2022] DELETE /identity/v3/projects/3774701788374f28b460244f98d683dc => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:27:13.011076 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01883e-1feb-4444-8ab7-6736a1763081 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:13.011368 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01883e-1feb-4444-8ab7-6736a1763081 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:13.011673 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01883e-1feb-4444-8ab7-6736a1763081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:13.028161 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c4a51-63cb-42cb-8e6e-7c32ed1de1b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:13.028873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c4a51-63cb-42cb-8e6e-7c32ed1de1b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:13.031205 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c4a51-63cb-42cb-8e6e-7c32ed1de1b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:13.031205 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c4a51-63cb-42cb-8e6e-7c32ed1de1b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:13.031440 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-9e01883e-1feb-4444-8ab7-6736a1763081 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:13.031558 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-716c4a51-63cb-42cb-8e6e-7c32ed1de1b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:13.032038 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-9e01883e-1feb-4444-8ab7-6736a1763081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:13.094107 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1336/2639] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:27:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:27:13.096139 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716c4a51-63cb-42cb-8e6e-7c32ed1de1b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:13.097587 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716c4a51-63cb-42cb-8e6e-7c32ed1de1b1 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:13.101714 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ec6c6-4b46-4a2b-b710-887a821943c1 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:13.107537 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ec6c6-4b46-4a2b-b710-887a821943c1 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:13.107537 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ec6c6-4b46-4a2b-b710-887a821943c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:13.108761 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1304/2640] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:27:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:27:13.123919 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-2e6ec6c6-4b46-4a2b-b710-887a821943c1 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:13.125193 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-2e6ec6c6-4b46-4a2b-b710-887a821943c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:13.135470 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1337/2641] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:27:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:27:13.139799 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c4a51-63cb-42cb-8e6e-7c32ed1de1b1 service neutron] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:13.149380 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c4a51-63cb-42cb-8e6e-7c32ed1de1b1 service neutron] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:13.155304 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680897ce-8145-49f9-a81d-d159d0860186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:13.155919 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680897ce-8145-49f9-a81d-d159d0860186 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:13.156206 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680897ce-8145-49f9-a81d-d159d0860186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:13.156486 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680897ce-8145-49f9-a81d-d159d0860186 admin admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:13.157144 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680897ce-8145-49f9-a81d-d159d0860186 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:27:13.157456 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680897ce-8145-49f9-a81d-d159d0860186 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:13.159052 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680897ce-8145-49f9-a81d-d159d0860186 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:13.172291 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1305/2642] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:27:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:27:13.176852 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ec6c6-4b46-4a2b-b710-887a821943c1 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:13.186474 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ec6c6-4b46-4a2b-b710-887a821943c1 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:13.191470 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a847fb5-f76a-48e8-a59f-8203f36129ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:13.195618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a847fb5-f76a-48e8-a59f-8203f36129ed admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:13.195618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a847fb5-f76a-48e8-a59f-8203f36129ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:13.195618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a847fb5-f76a-48e8-a59f-8203f36129ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:13.195618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a847fb5-f76a-48e8-a59f-8203f36129ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:13.195618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a847fb5-f76a-48e8-a59f-8203f36129ed admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:13.290551 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1338/2643] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:27:13 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:27:13.291879 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680897ce-8145-49f9-a81d-d159d0860186 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:13.304602 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680897ce-8145-49f9-a81d-d159d0860186 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:13.365379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1373032-c342-4e8b-8254-580099691288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:13.366005 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1373032-c342-4e8b-8254-580099691288 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:13.366287 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1373032-c342-4e8b-8254-580099691288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:13.366561 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1373032-c342-4e8b-8254-580099691288 admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:13.367200 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1373032-c342-4e8b-8254-580099691288 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:13.369182 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1373032-c342-4e8b-8254-580099691288 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:13.469757 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1306/2644] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:27:13 2022] POST /identity/v3/users => generated 292 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:27:13.473839 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a847fb5-f76a-48e8-a59f-8203f36129ed admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:13.481353 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:27:13.485484 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a847fb5-f76a-48e8-a59f-8203f36129ed admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:13.490746 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d17e78-6019-4244-8182-224c3fba3d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:13.497796 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d17e78-6019-4244-8182-224c3fba3d2c admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:13.497796 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d17e78-6019-4244-8182-224c3fba3d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:13.497796 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d17e78-6019-4244-8182-224c3fba3d2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:13.497796 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d17e78-6019-4244-8182-224c3fba3d2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:13.497796 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d17e78-6019-4244-8182-224c3fba3d2c admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:13.500252 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1373032-c342-4e8b-8254-580099691288 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:13.502167 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-a1373032-c342-4e8b-8254-580099691288 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:13.506509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1339/2645] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:27:13.510471 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d17e78-6019-4244-8182-224c3fba3d2c admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:13.522623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d17e78-6019-4244-8182-224c3fba3d2c admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:13.523225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:27:13.530207 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1373032-c342-4e8b-8254-580099691288 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:13.530377 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27f370-a1f1-4ae5-a03b-3709577b7437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:13.530977 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27f370-a1f1-4ae5-a03b-3709577b7437 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:13.531257 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27f370-a1f1-4ae5-a03b-3709577b7437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:13.531593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27f370-a1f1-4ae5-a03b-3709577b7437 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:13.532273 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa27f370-a1f1-4ae5-a03b-3709577b7437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:13.533924 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa27f370-a1f1-4ae5-a03b-3709577b7437 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:13.535103 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda754eb-ce67-4058-82ef-470f94bb727c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:13.536065 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda754eb-ce67-4058-82ef-470f94bb727c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:13.536220 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda754eb-ce67-4058-82ef-470f94bb727c admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:13.536404 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda754eb-ce67-4058-82ef-470f94bb727c admin admin] PATH_INFO: `/v3/projects/00b03fe7e2354b919129088ef447ce2e` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:13.537561 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda754eb-ce67-4058-82ef-470f94bb727c admin admin] RBAC: Authorizing `identity:delete_project(project_id=00b03fe7e2354b919129088ef447ce2e)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:13.539525 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda754eb-ce67-4058-82ef-470f94bb727c admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:13.545605 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1340/2646] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 12:27:13.551145 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27f370-a1f1-4ae5-a03b-3709577b7437 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:13.563753 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27f370-a1f1-4ae5-a03b-3709577b7437 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:13.569259 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b605b7-c331-40df-9547-4fe2aea26fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:13.569991 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b605b7-c331-40df-9547-4fe2aea26fa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:13.570303 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b605b7-c331-40df-9547-4fe2aea26fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:13.581196 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b605b7-c331-40df-9547-4fe2aea26fa1 admin admin] PATH_INFO: `/v3/projects/9ccee40919ee4a53998a65eb3841bd5c/users/63b2ac068a3e409ab57fa6bf12d3b1d1/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:13.586195 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b605b7-c331-40df-9547-4fe2aea26fa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ccee40919ee4a53998a65eb3841bd5c, user_id=63b2ac068a3e409ab57fa6bf12d3b1d1, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:13.590278 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b605b7-c331-40df-9547-4fe2aea26fa1 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:13.712506 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1341/2647] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:27:13 2022] PUT /identity/v3/projects/9ccee40919ee4a53998a65eb3841bd5c/users/63b2ac068a3e409ab57fa6bf12d3b1d1/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:27:13.729708 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-dda754eb-ce67-4058-82ef-470f94bb727c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00b03fe7e2354b919129088ef447ce2e', 'request_id': 'req-dda754eb-ce67-4058-82ef-470f94bb727c'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:13.729890 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84edba-e9d0-4546-91f9-5820591b89a2 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:13.729890 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84edba-e9d0-4546-91f9-5820591b89a2 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:13.729890 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84edba-e9d0-4546-91f9-5820591b89a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:13.730406 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-dda754eb-ce67-4058-82ef-470f94bb727c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00b03fe7e2354b919129088ef447ce2e', 'request_id': 'req-dda754eb-ce67-4058-82ef-470f94bb727c'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:13.752390 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-5a84edba-e9d0-4546-91f9-5820591b89a2 None None] MFA Rules not processed for user `63b2ac068a3e409ab57fa6bf12d3b1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:13.753695 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-5a84edba-e9d0-4546-91f9-5820591b89a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:13.805293 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-dda754eb-ce67-4058-82ef-470f94bb727c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00b03fe7e2354b919129088ef447ce2e', 'request_id': 'req-dda754eb-ce67-4058-82ef-470f94bb727c'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:13.818677 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1342/2648] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:27:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:27:13.820375 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-dda754eb-ce67-4058-82ef-470f94bb727c admin admin] The token cache is being invalidate because project 00b03fe7e2354b919129088ef447ce2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:27:13.820730 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-dda754eb-ce67-4058-82ef-470f94bb727c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:13.822114 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-dda754eb-ce67-4058-82ef-470f94bb727c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:13.823808 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1307/2649] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:27:13 2022] DELETE /identity/v3/projects/00b03fe7e2354b919129088ef447ce2e => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:27:13.828374 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375cfdaa-9f07-480d-bbfb-f45d1dcb55e5 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:13.828777 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375cfdaa-9f07-480d-bbfb-f45d1dcb55e5 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:13.829154 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375cfdaa-9f07-480d-bbfb-f45d1dcb55e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:13.830426 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda754eb-ce67-4058-82ef-470f94bb727c admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:13.832226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-dda754eb-ce67-4058-82ef-470f94bb727c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:13.851251 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-375cfdaa-9f07-480d-bbfb-f45d1dcb55e5 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:13.853093 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-375cfdaa-9f07-480d-bbfb-f45d1dcb55e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:13.897854 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda754eb-ce67-4058-82ef-470f94bb727c admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:13.902855 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f2d119-9622-402c-a90f-c0bb530c92e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:13.903584 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f2d119-9622-402c-a90f-c0bb530c92e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:13.904363 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f2d119-9622-402c-a90f-c0bb530c92e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:13.904363 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f2d119-9622-402c-a90f-c0bb530c92e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:13.906175 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-f3f2d119-9622-402c-a90f-c0bb530c92e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:13.918975 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f2d119-9622-402c-a90f-c0bb530c92e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:13.920452 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f2d119-9622-402c-a90f-c0bb530c92e8 service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:13.923672 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1343/2650] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:27:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:27:13.930408 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1308/2651] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:27:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:27:13.932285 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28705b56-5ea9-4d6b-b60b-62472760d055 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:13.932623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28705b56-5ea9-4d6b-b60b-62472760d055 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:13.932908 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28705b56-5ea9-4d6b-b60b-62472760d055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:13.953107 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-28705b56-5ea9-4d6b-b60b-62472760d055 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:13.954339 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-28705b56-5ea9-4d6b-b60b-62472760d055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:13.966056 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1344/2652] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:27:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:27:13.967300 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f2d119-9622-402c-a90f-c0bb530c92e8 service cinder] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:13.978764 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f2d119-9622-402c-a90f-c0bb530c92e8 service cinder] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:13.984882 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6629ca8-df0f-43e2-9b3f-e13c2dd99507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:13.985426 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6629ca8-df0f-43e2-9b3f-e13c2dd99507 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:13.985707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6629ca8-df0f-43e2-9b3f-e13c2dd99507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:13.985980 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6629ca8-df0f-43e2-9b3f-e13c2dd99507 admin admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:13.986633 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6629ca8-df0f-43e2-9b3f-e13c2dd99507 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:27:13.986932 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6629ca8-df0f-43e2-9b3f-e13c2dd99507 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:13.988560 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6629ca8-df0f-43e2-9b3f-e13c2dd99507 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:14.006008 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1309/2653] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:27:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:27:14.010668 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28705b56-5ea9-4d6b-b60b-62472760d055 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:14.020216 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28705b56-5ea9-4d6b-b60b-62472760d055 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:14.024997 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e1269-4c3d-451a-9aae-291bcd64cbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:14.025609 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e1269-4c3d-451a-9aae-291bcd64cbb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:14.026044 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e1269-4c3d-451a-9aae-291bcd64cbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:14.026875 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e1269-4c3d-451a-9aae-291bcd64cbb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:14.027561 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e1269-4c3d-451a-9aae-291bcd64cbb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:14.029506 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e1269-4c3d-451a-9aae-291bcd64cbb5 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:14.150149 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1345/2654] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:27:14 2022] POST /identity/v3/projects => generated 369 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:27:14.154972 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6629ca8-df0f-43e2-9b3f-e13c2dd99507 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:14.165775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6629ca8-df0f-43e2-9b3f-e13c2dd99507 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:14.233880 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ea4ab-043d-4d6c-8551-c0adb6e1068a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:14.234484 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ea4ab-043d-4d6c-8551-c0adb6e1068a admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:14.234761 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ea4ab-043d-4d6c-8551-c0adb6e1068a admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:14.235084 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ea4ab-043d-4d6c-8551-c0adb6e1068a admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:14.235690 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ea4ab-043d-4d6c-8551-c0adb6e1068a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:14.237582 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ea4ab-043d-4d6c-8551-c0adb6e1068a admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:14.343638 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1310/2655] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:27:14 2022] POST /identity/v3/users => generated 285 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:27:14.347842 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e1269-4c3d-451a-9aae-291bcd64cbb5 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:14.358335 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e1269-4c3d-451a-9aae-291bcd64cbb5 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:14.362988 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db8dab-5c64-4cb1-ae64-e6ec5c006e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:14.364365 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db8dab-5c64-4cb1-ae64-e6ec5c006e7f admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:14.364365 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db8dab-5c64-4cb1-ae64-e6ec5c006e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:14.364365 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db8dab-5c64-4cb1-ae64-e6ec5c006e7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:14.364811 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db8dab-5c64-4cb1-ae64-e6ec5c006e7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:14.366429 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db8dab-5c64-4cb1-ae64-e6ec5c006e7f admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:14.376907 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1346/2656] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:27:14.384629 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ea4ab-043d-4d6c-8551-c0adb6e1068a admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:14.391679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:27:14.395595 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ea4ab-043d-4d6c-8551-c0adb6e1068a admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:14.402241 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c4c59-7d78-4558-9be4-e27eb4253090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:14.402849 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c4c59-7d78-4558-9be4-e27eb4253090 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:14.403134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c4c59-7d78-4558-9be4-e27eb4253090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:14.403724 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c4c59-7d78-4558-9be4-e27eb4253090 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:14.404088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516c4c59-7d78-4558-9be4-e27eb4253090 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:14.405675 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516c4c59-7d78-4558-9be4-e27eb4253090 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:14.417428 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1311/2657] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:27:14.421556 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db8dab-5c64-4cb1-ae64-e6ec5c006e7f admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:14.432114 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db8dab-5c64-4cb1-ae64-e6ec5c006e7f admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:14.436765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e5f4e9-c8f3-4599-91c2-8a88d7e48ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:14.437415 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5f4e9-c8f3-4599-91c2-8a88d7e48ee6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:14.437700 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5f4e9-c8f3-4599-91c2-8a88d7e48ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:14.437978 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5f4e9-c8f3-4599-91c2-8a88d7e48ee6 admin admin] PATH_INFO: `/v3/projects/672f308b487a4c0bb2d3e731542289cf/users/cb91581b839c4318a1bb6c53e63f2649/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:14.441136 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:27:14.453427 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e5f4e9-c8f3-4599-91c2-8a88d7e48ee6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=672f308b487a4c0bb2d3e731542289cf, user_id=cb91581b839c4318a1bb6c53e63f2649, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:14.456176 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e5f4e9-c8f3-4599-91c2-8a88d7e48ee6 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:14.537105 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1347/2658] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:27:14 2022] PUT /identity/v3/projects/672f308b487a4c0bb2d3e731542289cf/users/cb91581b839c4318a1bb6c53e63f2649/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:27:14.542709 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd53bd-c9d2-4813-800c-25d8bd8aabec None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:14.543015 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd53bd-c9d2-4813-800c-25d8bd8aabec None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:14.543292 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd53bd-c9d2-4813-800c-25d8bd8aabec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:14.563204 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-35cd53bd-c9d2-4813-800c-25d8bd8aabec None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:14.564579 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-35cd53bd-c9d2-4813-800c-25d8bd8aabec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:14.631742 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1312/2659] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:27:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:27:14.641997 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e5f4e9-c8f3-4599-91c2-8a88d7e48ee6 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:14.643728 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-16e5f4e9-c8f3-4599-91c2-8a88d7e48ee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:14.707756 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e5f4e9-c8f3-4599-91c2-8a88d7e48ee6 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:14.715529 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d646692f-8c93-4fd3-befd-5ac246e743bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:14.715529 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646692f-8c93-4fd3-befd-5ac246e743bb service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:14.715529 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646692f-8c93-4fd3-befd-5ac246e743bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:14.715529 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646692f-8c93-4fd3-befd-5ac246e743bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:14.724330 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d646692f-8c93-4fd3-befd-5ac246e743bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:14.725773 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d646692f-8c93-4fd3-befd-5ac246e743bb service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:14.737200 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1348/2660] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:27:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:27:15.987614 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797e6e1-62ce-4508-bb95-09389674f5bc None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:15.988182 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797e6e1-62ce-4508-bb95-09389674f5bc None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:15.988455 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797e6e1-62ce-4508-bb95-09389674f5bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:16.023131 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-6797e6e1-62ce-4508-bb95-09389674f5bc None None] MFA Rules not processed for user `cb91581b839c4318a1bb6c53e63f2649`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:16.024400 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-6797e6e1-62ce-4508-bb95-09389674f5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:16.093562 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1313/2661] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:27:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:27:16.104253 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d646692f-8c93-4fd3-befd-5ac246e743bb service neutron] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:16.172366 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d646692f-8c93-4fd3-befd-5ac246e743bb service neutron] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:16.175991 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3dcd47-6f38-4a56-b5da-44815c8065c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:16.176605 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3dcd47-6f38-4a56-b5da-44815c8065c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:16.176903 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3dcd47-6f38-4a56-b5da-44815c8065c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:16.177166 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3dcd47-6f38-4a56-b5da-44815c8065c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:16.186184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3dcd47-6f38-4a56-b5da-44815c8065c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:16.188725 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3dcd47-6f38-4a56-b5da-44815c8065c1 service cinder] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:16.198360 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1349/2662] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:27:16 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:27:20.148394 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba9150-7862-43a6-a7ed-51c82e02b5b5 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:20.148988 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba9150-7862-43a6-a7ed-51c82e02b5b5 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:20.148988 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba9150-7862-43a6-a7ed-51c82e02b5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:20.167606 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-a5ba9150-7862-43a6-a7ed-51c82e02b5b5 None None] MFA Rules not processed for user `cb91581b839c4318a1bb6c53e63f2649`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:20.168815 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-a5ba9150-7862-43a6-a7ed-51c82e02b5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:20.176244 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1314/2663] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:27:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:27:22.834261 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3dcd47-6f38-4a56-b5da-44815c8065c1 service cinder] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:22.836031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-7c3dcd47-6f38-4a56-b5da-44815c8065c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:22.850375 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3dcd47-6f38-4a56-b5da-44815c8065c1 service cinder] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:22.855229 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d761e3-158f-487f-bd3e-6f271e9a6818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:22.855901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d761e3-158f-487f-bd3e-6f271e9a6818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:22.856206 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d761e3-158f-487f-bd3e-6f271e9a6818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:22.856485 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d761e3-158f-487f-bd3e-6f271e9a6818 admin admin] PATH_INFO: `/v3/users/63b2ac068a3e409ab57fa6bf12d3b1d1` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:22.857680 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d761e3-158f-487f-bd3e-6f271e9a6818 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63b2ac068a3e409ab57fa6bf12d3b1d1)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:22.859634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d761e3-158f-487f-bd3e-6f271e9a6818 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:23.077149 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-d0d761e3-158f-487f-bd3e-6f271e9a6818 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63b2ac068a3e409ab57fa6bf12d3b1d1'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:23.084654 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-d0d761e3-158f-487f-bd3e-6f271e9a6818 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63b2ac068a3e409ab57fa6bf12d3b1d1'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:23.085345 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-d0d761e3-158f-487f-bd3e-6f271e9a6818 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63b2ac068a3e409ab57fa6bf12d3b1d1'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:23.098080 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-d0d761e3-158f-487f-bd3e-6f271e9a6818 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63b2ac068a3e409ab57fa6bf12d3b1d1'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:23.200009 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-d0d761e3-158f-487f-bd3e-6f271e9a6818 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63b2ac068a3e409ab57fa6bf12d3b1d1'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:23.207008 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1350/2664] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:27:22 2022] DELETE /identity/v3/users/63b2ac068a3e409ab57fa6bf12d3b1d1 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:27:23.211408 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59559b38-be0f-49a4-ac4f-9b277b1a77a1 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:23.211952 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59559b38-be0f-49a4-ac4f-9b277b1a77a1 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:23.212067 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59559b38-be0f-49a4-ac4f-9b277b1a77a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:23.231118 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-59559b38-be0f-49a4-ac4f-9b277b1a77a1 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:23.232894 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-59559b38-be0f-49a4-ac4f-9b277b1a77a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:23.304146 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1315/2665] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:27:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:27:23.316607 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d761e3-158f-487f-bd3e-6f271e9a6818 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:23.318315 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-d0d761e3-158f-487f-bd3e-6f271e9a6818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:23.325531 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:27:23.392798 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d761e3-158f-487f-bd3e-6f271e9a6818 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:23.396648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870bffbb-91b5-4d96-89e7-c495ede3564e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:23.397275 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870bffbb-91b5-4d96-89e7-c495ede3564e service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:23.397552 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870bffbb-91b5-4d96-89e7-c495ede3564e service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:23.397824 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870bffbb-91b5-4d96-89e7-c495ede3564e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:23.407621 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870bffbb-91b5-4d96-89e7-c495ede3564e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:23.409055 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870bffbb-91b5-4d96-89e7-c495ede3564e service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:23.422747 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1351/2666] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:27:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:27:24.050763 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59559b38-be0f-49a4-ac4f-9b277b1a77a1 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:24.052443 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-59559b38-be0f-49a4-ac4f-9b277b1a77a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:24.077925 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59559b38-be0f-49a4-ac4f-9b277b1a77a1 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:24.085038 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0160f87-cec5-4c0b-b8db-d024a45fdb4f None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:24.085856 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0160f87-cec5-4c0b-b8db-d024a45fdb4f None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:24.086398 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0160f87-cec5-4c0b-b8db-d024a45fdb4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:24.087209 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b1452-6338-4224-9f19-8fc86b1adc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:24.088219 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b1452-6338-4224-9f19-8fc86b1adc72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:24.088592 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b1452-6338-4224-9f19-8fc86b1adc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:24.088979 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b1452-6338-4224-9f19-8fc86b1adc72 admin admin] PATH_INFO: `/v3/projects/9ccee40919ee4a53998a65eb3841bd5c` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:24.093415 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9b1452-6338-4224-9f19-8fc86b1adc72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ccee40919ee4a53998a65eb3841bd5c)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:24.095678 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9b1452-6338-4224-9f19-8fc86b1adc72 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:24.114164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:27:24.126244 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-e0160f87-cec5-4c0b-b8db-d024a45fdb4f None None] MFA Rules not processed for user `b6880278f7304051b9ef03e927637e74`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:24.128295 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-e0160f87-cec5-4c0b-b8db-d024a45fdb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:24.191161 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1352/2667] 10.222.0.98 () {58 vars in 1009 bytes} [Thu Aug 18 12:27:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:27:24.254708 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-fa9b1452-6338-4224-9f19-8fc86b1adc72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ccee40919ee4a53998a65eb3841bd5c', 'request_id': 'req-fa9b1452-6338-4224-9f19-8fc86b1adc72'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:24.262320 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-fa9b1452-6338-4224-9f19-8fc86b1adc72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ccee40919ee4a53998a65eb3841bd5c', 'request_id': 'req-fa9b1452-6338-4224-9f19-8fc86b1adc72'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:24.333588 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-fa9b1452-6338-4224-9f19-8fc86b1adc72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ccee40919ee4a53998a65eb3841bd5c', 'request_id': 'req-fa9b1452-6338-4224-9f19-8fc86b1adc72'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:24.345274 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0160f87-cec5-4c0b-b8db-d024a45fdb4f None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:24.346964 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-e0160f87-cec5-4c0b-b8db-d024a45fdb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:24.349182 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-fa9b1452-6338-4224-9f19-8fc86b1adc72 admin admin] The token cache is being invalidate because project 9ccee40919ee4a53998a65eb3841bd5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:27:24.349841 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-fa9b1452-6338-4224-9f19-8fc86b1adc72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:24.350701 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-fa9b1452-6338-4224-9f19-8fc86b1adc72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:24.357088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1316/2668] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:27:24 2022] DELETE /identity/v3/projects/9ccee40919ee4a53998a65eb3841bd5c => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:27:24.358623 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ecf9de-a18a-4e05-905f-dbe86afc9ad7 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:24.359083 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ecf9de-a18a-4e05-905f-dbe86afc9ad7 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:24.359566 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ecf9de-a18a-4e05-905f-dbe86afc9ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:24.384512 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-00ecf9de-a18a-4e05-905f-dbe86afc9ad7 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:24.385782 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-00ecf9de-a18a-4e05-905f-dbe86afc9ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:24.432253 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0160f87-cec5-4c0b-b8db-d024a45fdb4f None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:24.482050 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1317/2669] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:27:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:27:24.485128 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad639ee3-3afb-4785-9985-00a661908416 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:24.485552 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad639ee3-3afb-4785-9985-00a661908416 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:24.485931 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad639ee3-3afb-4785-9985-00a661908416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:24.509116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-ad639ee3-3afb-4785-9985-00a661908416 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:24.510479 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-ad639ee3-3afb-4785-9985-00a661908416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:24.520113 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1318/2670] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:27:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:27:24.523741 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:27:24.526025 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad639ee3-3afb-4785-9985-00a661908416 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:24.543480 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43678e3a-accd-4f3a-a37d-a5b2e121a2bb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771591fe31cb4e8c9c4320c2a7b2719d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:24.544479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43678e3a-accd-4f3a-a37d-a5b2e121a2bb service swift] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:24.544982 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43678e3a-accd-4f3a-a37d-a5b2e121a2bb service swift] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:24.545435 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43678e3a-accd-4f3a-a37d-a5b2e121a2bb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:24.548692 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-43678e3a-accd-4f3a-a37d-a5b2e121a2bb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:24.549906 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad639ee3-3afb-4785-9985-00a661908416 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:24.557383 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a44dc-0427-4332-8df0-152e061ff86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:24.558350 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a44dc-0427-4332-8df0-152e061ff86a admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:24.558881 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a44dc-0427-4332-8df0-152e061ff86a admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:24.560948 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a44dc-0427-4332-8df0-152e061ff86a admin admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:24.561361 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a44dc-0427-4332-8df0-152e061ff86a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:27:24.561962 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a44dc-0427-4332-8df0-152e061ff86a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:24.565090 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a44dc-0427-4332-8df0-152e061ff86a admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:24.584722 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1319/2671] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:27:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:27:24.589123 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a44dc-0427-4332-8df0-152e061ff86a admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:24.599480 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a44dc-0427-4332-8df0-152e061ff86a admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:24.607610 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd6aef-48e4-42d3-a62b-de9434cc7fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:24.608486 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd6aef-48e4-42d3-a62b-de9434cc7fa8 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:24.609024 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd6aef-48e4-42d3-a62b-de9434cc7fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:24.609261 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd6aef-48e4-42d3-a62b-de9434cc7fa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:24.609957 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd6aef-48e4-42d3-a62b-de9434cc7fa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:24.612096 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd6aef-48e4-42d3-a62b-de9434cc7fa8 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:24.631821 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43678e3a-accd-4f3a-a37d-a5b2e121a2bb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:24.633457 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43678e3a-accd-4f3a-a37d-a5b2e121a2bb service swift] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:24.642914 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1353/2672] 10.222.0.98 () {60 vars in 1384 bytes} [Thu Aug 18 12:27:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 18 12:27:24.689390 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1320/2673] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:27:24 2022] POST /identity/v3/projects => generated 371 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:27:24.694341 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43678e3a-accd-4f3a-a37d-a5b2e121a2bb service swift] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:24.706995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43678e3a-accd-4f3a-a37d-a5b2e121a2bb service swift] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:24.791362 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c9a032-257b-4383-bb5f-d6d8ecfc39ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:24.792039 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9a032-257b-4383-bb5f-d6d8ecfc39ea admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:24.792395 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9a032-257b-4383-bb5f-d6d8ecfc39ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:24.792748 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9a032-257b-4383-bb5f-d6d8ecfc39ea admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:24.793529 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c9a032-257b-4383-bb5f-d6d8ecfc39ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:24.796100 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c9a032-257b-4383-bb5f-d6d8ecfc39ea admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:24.907714 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1354/2674] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:27:24 2022] POST /identity/v3/users => generated 286 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:27:24.912707 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd6aef-48e4-42d3-a62b-de9434cc7fa8 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:24.925795 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd6aef-48e4-42d3-a62b-de9434cc7fa8 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:24.936974 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d768c562-24a1-4be2-9bc6-1b300fded985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:24.937667 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768c562-24a1-4be2-9bc6-1b300fded985 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:24.937960 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768c562-24a1-4be2-9bc6-1b300fded985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:24.938234 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768c562-24a1-4be2-9bc6-1b300fded985 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:24.938890 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d768c562-24a1-4be2-9bc6-1b300fded985 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:24.940892 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d768c562-24a1-4be2-9bc6-1b300fded985 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:24.960843 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c9a032-257b-4383-bb5f-d6d8ecfc39ea admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:24.969577 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1321/2675] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:27:24.975019 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c9a032-257b-4383-bb5f-d6d8ecfc39ea admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:24.985272 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618bcf66-57b9-48f2-95c0-371109036ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:24.985906 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618bcf66-57b9-48f2-95c0-371109036ebf admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:24.986201 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618bcf66-57b9-48f2-95c0-371109036ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:24.986479 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618bcf66-57b9-48f2-95c0-371109036ebf admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:24.987147 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618bcf66-57b9-48f2-95c0-371109036ebf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:24.988875 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618bcf66-57b9-48f2-95c0-371109036ebf admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:25.008617 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1355/2676] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:27:25.008722 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d768c562-24a1-4be2-9bc6-1b300fded985 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:25.019450 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d768c562-24a1-4be2-9bc6-1b300fded985 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:25.026881 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48021c6c-babf-4728-98a8-853a48ece337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:25.028000 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48021c6c-babf-4728-98a8-853a48ece337 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:25.028486 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48021c6c-babf-4728-98a8-853a48ece337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:25.028769 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48021c6c-babf-4728-98a8-853a48ece337 admin admin] PATH_INFO: `/v3/projects/440acad60ac241f4909a354339a878ef/users/97623baf82b3470ca5679619642fe562/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:25.047200 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48021c6c-babf-4728-98a8-853a48ece337 admin admin] RBAC: Authorizing `identity:create_grant(project_id=440acad60ac241f4909a354339a878ef, user_id=97623baf82b3470ca5679619642fe562, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:25.050061 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48021c6c-babf-4728-98a8-853a48ece337 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:25.174280 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1322/2677] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:27:25 2022] PUT /identity/v3/projects/440acad60ac241f4909a354339a878ef/users/97623baf82b3470ca5679619642fe562/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:27:25.189026 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d3353-135b-4fe9-92e0-fdc6f7d5dbed None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:25.189638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d3353-135b-4fe9-92e0-fdc6f7d5dbed None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:25.189814 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d3353-135b-4fe9-92e0-fdc6f7d5dbed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:25.238723 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-ab3d3353-135b-4fe9-92e0-fdc6f7d5dbed None None] MFA Rules not processed for user `97623baf82b3470ca5679619642fe562`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:25.240421 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-ab3d3353-135b-4fe9-92e0-fdc6f7d5dbed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:25.376131 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1356/2678] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:27:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:27:25.388165 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48021c6c-babf-4728-98a8-853a48ece337 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:25.390302 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-48021c6c-babf-4728-98a8-853a48ece337 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:25.438576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:27:25.489079 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48021c6c-babf-4728-98a8-853a48ece337 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:25.493067 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6751de76-a546-47d1-8ad0-42c7edfd6e51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:25.493931 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751de76-a546-47d1-8ad0-42c7edfd6e51 service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:25.494253 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751de76-a546-47d1-8ad0-42c7edfd6e51 service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:25.494539 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751de76-a546-47d1-8ad0-42c7edfd6e51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:25.507640 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6751de76-a546-47d1-8ad0-42c7edfd6e51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:25.509624 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6751de76-a546-47d1-8ad0-42c7edfd6e51 service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:25.537396 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1323/2679] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:27:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:27:25.557502 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3d3353-135b-4fe9-92e0-fdc6f7d5dbed None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:25.560499 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:27:25.651792 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3d3353-135b-4fe9-92e0-fdc6f7d5dbed None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:25.658208 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be5c31-87d0-4fe4-b4df-166105c4f373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:25.658947 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be5c31-87d0-4fe4-b4df-166105c4f373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:25.659299 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be5c31-87d0-4fe4-b4df-166105c4f373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:25.659721 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be5c31-87d0-4fe4-b4df-166105c4f373 admin admin] PATH_INFO: `/v3/users/97623baf82b3470ca5679619642fe562` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:25.660918 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be5c31-87d0-4fe4-b4df-166105c4f373 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97623baf82b3470ca5679619642fe562)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:25.663143 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be5c31-87d0-4fe4-b4df-166105c4f373 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:26.087231 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-39be5c31-87d0-4fe4-b4df-166105c4f373 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97623baf82b3470ca5679619642fe562'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:26.100916 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-39be5c31-87d0-4fe4-b4df-166105c4f373 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97623baf82b3470ca5679619642fe562'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:26.102653 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-39be5c31-87d0-4fe4-b4df-166105c4f373 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97623baf82b3470ca5679619642fe562'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:26.120917 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-39be5c31-87d0-4fe4-b4df-166105c4f373 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97623baf82b3470ca5679619642fe562'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:26.362368 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-39be5c31-87d0-4fe4-b4df-166105c4f373 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97623baf82b3470ca5679619642fe562'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:26.373044 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1357/2680] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:27:25 2022] DELETE /identity/v3/users/97623baf82b3470ca5679619642fe562 => generated 0 bytes in 816 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:27:26.375027 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf35deb-a5c1-4ce2-9c35-36c95585bcf3 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:26.375392 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf35deb-a5c1-4ce2-9c35-36c95585bcf3 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:26.375874 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf35deb-a5c1-4ce2-9c35-36c95585bcf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:26.397337 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-cdf35deb-a5c1-4ce2-9c35-36c95585bcf3 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:26.398757 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-cdf35deb-a5c1-4ce2-9c35-36c95585bcf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:26.489667 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1324/2681] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:27:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:27:26.498094 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be5c31-87d0-4fe4-b4df-166105c4f373 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:26.499697 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-39be5c31-87d0-4fe4-b4df-166105c4f373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:26.575655 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be5c31-87d0-4fe4-b4df-166105c4f373 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:26.582547 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf085d9a-db26-477c-b012-10fa5bcd27f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:26.583301 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf085d9a-db26-477c-b012-10fa5bcd27f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:26.583644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf085d9a-db26-477c-b012-10fa5bcd27f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:26.584000 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf085d9a-db26-477c-b012-10fa5bcd27f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:26.597080 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf085d9a-db26-477c-b012-10fa5bcd27f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:26.598813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf085d9a-db26-477c-b012-10fa5bcd27f7 service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:26.622162 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1358/2682] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:27:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:27:27.103740 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf35deb-a5c1-4ce2-9c35-36c95585bcf3 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:27.106096 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-cdf35deb-a5c1-4ce2-9c35-36c95585bcf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:27.124653 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf35deb-a5c1-4ce2-9c35-36c95585bcf3 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:27.130636 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d9e87-9086-41fe-bf9f-28ce29b20804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:27.130873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d9e87-9086-41fe-bf9f-28ce29b20804 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:27.131212 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d9e87-9086-41fe-bf9f-28ce29b20804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:27.131404 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d9e87-9086-41fe-bf9f-28ce29b20804 admin admin] PATH_INFO: `/v3/projects/440acad60ac241f4909a354339a878ef` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:27.132521 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d9e87-9086-41fe-bf9f-28ce29b20804 admin admin] RBAC: Authorizing `identity:delete_project(project_id=440acad60ac241f4909a354339a878ef)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:27.134663 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d9e87-9086-41fe-bf9f-28ce29b20804 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:27.330492 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-f78d9e87-9086-41fe-bf9f-28ce29b20804 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '440acad60ac241f4909a354339a878ef', 'request_id': 'req-f78d9e87-9086-41fe-bf9f-28ce29b20804'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:27.340122 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-f78d9e87-9086-41fe-bf9f-28ce29b20804 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '440acad60ac241f4909a354339a878ef', 'request_id': 'req-f78d9e87-9086-41fe-bf9f-28ce29b20804'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:27.453490 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-f78d9e87-9086-41fe-bf9f-28ce29b20804 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '440acad60ac241f4909a354339a878ef', 'request_id': 'req-f78d9e87-9086-41fe-bf9f-28ce29b20804'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:27.466025 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-f78d9e87-9086-41fe-bf9f-28ce29b20804 admin admin] The token cache is being invalidate because project 440acad60ac241f4909a354339a878ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:27:27.466385 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-f78d9e87-9086-41fe-bf9f-28ce29b20804 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:27.467162 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-f78d9e87-9086-41fe-bf9f-28ce29b20804 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:27.472052 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1325/2683] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:27:27 2022] DELETE /identity/v3/projects/440acad60ac241f4909a354339a878ef => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:27:27.474003 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a63b4e-429d-4b4b-ae9b-bed3f0924336 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:27.474497 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a63b4e-429d-4b4b-ae9b-bed3f0924336 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:27.474684 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a63b4e-429d-4b4b-ae9b-bed3f0924336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:27.499018 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-30a63b4e-429d-4b4b-ae9b-bed3f0924336 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:27.500355 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-30a63b4e-429d-4b4b-ae9b-bed3f0924336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:27.521926 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:27:27.564030 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1359/2684] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:27:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:27:27.571873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda1021-f212-4e87-8e14-15b854ff2791 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:27.572173 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda1021-f212-4e87-8e14-15b854ff2791 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:27.572456 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda1021-f212-4e87-8e14-15b854ff2791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:27.595760 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-4fda1021-f212-4e87-8e14-15b854ff2791 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:27.597001 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-4fda1021-f212-4e87-8e14-15b854ff2791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:27.605329 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1326/2685] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:27:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:27:27.609835 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a63b4e-429d-4b4b-ae9b-bed3f0924336 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:27.619721 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a63b4e-429d-4b4b-ae9b-bed3f0924336 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:27.625057 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadd8eb1-ecaf-40d6-880c-803864da1ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:27.625532 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd8eb1-ecaf-40d6-880c-803864da1ace admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:27.625814 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd8eb1-ecaf-40d6-880c-803864da1ace admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:27.626094 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd8eb1-ecaf-40d6-880c-803864da1ace admin admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:27.626741 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadd8eb1-ecaf-40d6-880c-803864da1ace admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:27:27.627047 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadd8eb1-ecaf-40d6-880c-803864da1ace admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:27.628684 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadd8eb1-ecaf-40d6-880c-803864da1ace admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:27.641118 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1360/2686] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:27:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:27:27.645630 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda1021-f212-4e87-8e14-15b854ff2791 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:27.655093 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda1021-f212-4e87-8e14-15b854ff2791 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:27.659911 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75672bdc-84b3-47d2-8ade-d2e6375a2286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:27.661492 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75672bdc-84b3-47d2-8ade-d2e6375a2286 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:27.661799 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75672bdc-84b3-47d2-8ade-d2e6375a2286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:27.662090 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75672bdc-84b3-47d2-8ade-d2e6375a2286 admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:27.662768 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75672bdc-84b3-47d2-8ade-d2e6375a2286 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:27.665087 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75672bdc-84b3-47d2-8ade-d2e6375a2286 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:28.140853 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1327/2687] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:27:27 2022] POST /identity/v3/projects => generated 373 bytes in 496 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:27:28.145082 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadd8eb1-ecaf-40d6-880c-803864da1ace admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:28.156075 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadd8eb1-ecaf-40d6-880c-803864da1ace admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:28.223548 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa2f695-8c18-4217-b347-9545a0ba36a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:28.227768 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2f695-8c18-4217-b347-9545a0ba36a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:28.227768 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2f695-8c18-4217-b347-9545a0ba36a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:28.227768 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2f695-8c18-4217-b347-9545a0ba36a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:28.227768 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa2f695-8c18-4217-b347-9545a0ba36a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:28.229041 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa2f695-8c18-4217-b347-9545a0ba36a6 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:28.336873 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1361/2688] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:27:28 2022] POST /identity/v3/users => generated 287 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:27:28.343238 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75672bdc-84b3-47d2-8ade-d2e6375a2286 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:28.357107 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75672bdc-84b3-47d2-8ade-d2e6375a2286 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:28.363450 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077aace-0adb-41f4-89ef-c7c4bb1a3bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:28.364301 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077aace-0adb-41f4-89ef-c7c4bb1a3bda admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:28.364588 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077aace-0adb-41f4-89ef-c7c4bb1a3bda admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:28.365050 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077aace-0adb-41f4-89ef-c7c4bb1a3bda admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:28.365716 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8077aace-0adb-41f4-89ef-c7c4bb1a3bda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:28.367310 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8077aace-0adb-41f4-89ef-c7c4bb1a3bda admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:28.385259 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa2f695-8c18-4217-b347-9545a0ba36a6 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:28.395169 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1328/2689] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:27:28.397459 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa2f695-8c18-4217-b347-9545a0ba36a6 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:28.405469 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c36869-82ef-495a-b6bf-d7e5b7e54670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:28.406553 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c36869-82ef-495a-b6bf-d7e5b7e54670 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:28.406832 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c36869-82ef-495a-b6bf-d7e5b7e54670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:28.407107 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c36869-82ef-495a-b6bf-d7e5b7e54670 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:28.407991 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c36869-82ef-495a-b6bf-d7e5b7e54670 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:28.409496 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c36869-82ef-495a-b6bf-d7e5b7e54670 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:28.447044 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1362/2690] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:27:28.450971 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077aace-0adb-41f4-89ef-c7c4bb1a3bda admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:28.465765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077aace-0adb-41f4-89ef-c7c4bb1a3bda admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:28.474842 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15fd2de-373c-4a87-ad82-cb300e7db2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:28.475704 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fd2de-373c-4a87-ad82-cb300e7db2b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:28.475999 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fd2de-373c-4a87-ad82-cb300e7db2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:28.476284 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fd2de-373c-4a87-ad82-cb300e7db2b9 admin admin] PATH_INFO: `/v3/projects/61bf0f07ab314b2ba4c86124cb6b9a8b/users/99748795f40540b589e4c3826f299955/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:28.487279 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:27:28.498374 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15fd2de-373c-4a87-ad82-cb300e7db2b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61bf0f07ab314b2ba4c86124cb6b9a8b, user_id=99748795f40540b589e4c3826f299955, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:28.501369 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15fd2de-373c-4a87-ad82-cb300e7db2b9 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:28.616979 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1329/2691] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:27:28 2022] PUT /identity/v3/projects/61bf0f07ab314b2ba4c86124cb6b9a8b/users/99748795f40540b589e4c3826f299955/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:27:28.629580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0b626-3aa1-474d-a090-0572a213e46f None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:28.629580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0b626-3aa1-474d-a090-0572a213e46f None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:28.629580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0b626-3aa1-474d-a090-0572a213e46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:28.664038 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-04c0b626-3aa1-474d-a090-0572a213e46f None None] MFA Rules not processed for user `99748795f40540b589e4c3826f299955`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:28.665798 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-04c0b626-3aa1-474d-a090-0572a213e46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:28.750447 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1363/2692] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:27:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:27:28.761357 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15fd2de-373c-4a87-ad82-cb300e7db2b9 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:28.765788 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-d15fd2de-373c-4a87-ad82-cb300e7db2b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:28.834336 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15fd2de-373c-4a87-ad82-cb300e7db2b9 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:28.838458 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed1466-a58f-4d3a-a1b0-32b3e451b449 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:28.839203 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed1466-a58f-4d3a-a1b0-32b3e451b449 service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:28.839525 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed1466-a58f-4d3a-a1b0-32b3e451b449 service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:28.839939 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed1466-a58f-4d3a-a1b0-32b3e451b449 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:28.852697 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed1466-a58f-4d3a-a1b0-32b3e451b449 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:28.854160 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed1466-a58f-4d3a-a1b0-32b3e451b449 service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:28.865722 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1330/2693] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:27:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:27:37.642410 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12300c-3d92-4af2-92e4-f9bb732fa5f2 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:37.643290 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12300c-3d92-4af2-92e4-f9bb732fa5f2 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:37.643758 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12300c-3d92-4af2-92e4-f9bb732fa5f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:37.662541 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-8e12300c-3d92-4af2-92e4-f9bb732fa5f2 None None] MFA Rules not processed for user `cb91581b839c4318a1bb6c53e63f2649`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:37.665128 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-8e12300c-3d92-4af2-92e4-f9bb732fa5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:37.677640 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:27:37.725617 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1364/2694] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:27:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:27:38.246646 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed1466-a58f-4d3a-a1b0-32b3e451b449 service cinder] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:38.248109 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-26ed1466-a58f-4d3a-a1b0-32b3e451b449 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:38.305818 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed1466-a58f-4d3a-a1b0-32b3e451b449 service cinder] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:38.309596 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960a63ea-9c77-4d33-ae94-0ace35c83bc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:38.310197 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a63ea-9c77-4d33-ae94-0ace35c83bc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:38.310483 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a63ea-9c77-4d33-ae94-0ace35c83bc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:38.310753 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a63ea-9c77-4d33-ae94-0ace35c83bc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:38.312622 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-960a63ea-9c77-4d33-ae94-0ace35c83bc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:38.322211 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960a63ea-9c77-4d33-ae94-0ace35c83bc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:38.323750 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960a63ea-9c77-4d33-ae94-0ace35c83bc1 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:38.332571 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1331/2695] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:27:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:27:41.302569 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e12300c-3d92-4af2-92e4-f9bb732fa5f2 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:41.368567 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e12300c-3d92-4af2-92e4-f9bb732fa5f2 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:41.373215 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe40a7-3013-476c-b67b-7fd744dde6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:41.373921 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe40a7-3013-476c-b67b-7fd744dde6a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:41.374272 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe40a7-3013-476c-b67b-7fd744dde6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:41.374542 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe40a7-3013-476c-b67b-7fd744dde6a1 admin admin] PATH_INFO: `/v3/users/99748795f40540b589e4c3826f299955` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:41.375762 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe40a7-3013-476c-b67b-7fd744dde6a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99748795f40540b589e4c3826f299955)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:41.379849 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe40a7-3013-476c-b67b-7fd744dde6a1 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:41.576726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-5dfe40a7-3013-476c-b67b-7fd744dde6a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99748795f40540b589e4c3826f299955'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:41.584494 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-5dfe40a7-3013-476c-b67b-7fd744dde6a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99748795f40540b589e4c3826f299955'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:41.585212 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-5dfe40a7-3013-476c-b67b-7fd744dde6a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99748795f40540b589e4c3826f299955'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:41.599363 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-5dfe40a7-3013-476c-b67b-7fd744dde6a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99748795f40540b589e4c3826f299955'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:41.675725 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-5dfe40a7-3013-476c-b67b-7fd744dde6a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99748795f40540b589e4c3826f299955'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:41.682636 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1365/2696] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:27:41 2022] DELETE /identity/v3/users/99748795f40540b589e4c3826f299955 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:27:41.687049 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3c1fc-00cd-42a1-bddb-e3c790c03ba6 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:41.687600 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3c1fc-00cd-42a1-bddb-e3c790c03ba6 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:41.687700 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3c1fc-00cd-42a1-bddb-e3c790c03ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:41.711372 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-7ce3c1fc-00cd-42a1-bddb-e3c790c03ba6 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:41.712630 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-7ce3c1fc-00cd-42a1-bddb-e3c790c03ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:41.780322 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1332/2697] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:27:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:27:41.790935 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe40a7-3013-476c-b67b-7fd744dde6a1 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:41.792426 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-5dfe40a7-3013-476c-b67b-7fd744dde6a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:41.854096 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:27:41.868808 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe40a7-3013-476c-b67b-7fd744dde6a1 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:41.872753 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe3645-ecf7-40b5-a0e5-d8f4248a149f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:41.873377 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe3645-ecf7-40b5-a0e5-d8f4248a149f service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:41.873668 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe3645-ecf7-40b5-a0e5-d8f4248a149f service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:41.873948 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe3645-ecf7-40b5-a0e5-d8f4248a149f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:41.884253 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe3645-ecf7-40b5-a0e5-d8f4248a149f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:41.886533 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe3645-ecf7-40b5-a0e5-d8f4248a149f service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:41.897251 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1366/2698] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:27:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:27:42.376311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce3c1fc-00cd-42a1-bddb-e3c790c03ba6 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:42.378254 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-7ce3c1fc-00cd-42a1-bddb-e3c790c03ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:42.392819 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce3c1fc-00cd-42a1-bddb-e3c790c03ba6 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:42.398021 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd153821-3667-476a-bdf4-3a00a66c4468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:42.398684 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd153821-3667-476a-bdf4-3a00a66c4468 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:42.399555 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd153821-3667-476a-bdf4-3a00a66c4468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:42.399555 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd153821-3667-476a-bdf4-3a00a66c4468 admin admin] PATH_INFO: `/v3/projects/61bf0f07ab314b2ba4c86124cb6b9a8b` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:42.400679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd153821-3667-476a-bdf4-3a00a66c4468 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61bf0f07ab314b2ba4c86124cb6b9a8b)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:42.402600 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd153821-3667-476a-bdf4-3a00a66c4468 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:42.551724 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:27:42.574978 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-cd153821-3667-476a-bdf4-3a00a66c4468 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61bf0f07ab314b2ba4c86124cb6b9a8b', 'request_id': 'req-cd153821-3667-476a-bdf4-3a00a66c4468'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:42.587330 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-cd153821-3667-476a-bdf4-3a00a66c4468 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61bf0f07ab314b2ba4c86124cb6b9a8b', 'request_id': 'req-cd153821-3667-476a-bdf4-3a00a66c4468'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:42.756088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-cd153821-3667-476a-bdf4-3a00a66c4468 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61bf0f07ab314b2ba4c86124cb6b9a8b', 'request_id': 'req-cd153821-3667-476a-bdf4-3a00a66c4468'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:42.768323 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-cd153821-3667-476a-bdf4-3a00a66c4468 admin admin] The token cache is being invalidate because project 61bf0f07ab314b2ba4c86124cb6b9a8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:27:42.768624 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-cd153821-3667-476a-bdf4-3a00a66c4468 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:42.769350 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-cd153821-3667-476a-bdf4-3a00a66c4468 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:27:42.770927 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1333/2699] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:27:42 2022] DELETE /identity/v3/projects/61bf0f07ab314b2ba4c86124cb6b9a8b => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:27:42.776645 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991c38a-aa56-4ee0-8c46-5014045a5e76 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:42.778695 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991c38a-aa56-4ee0-8c46-5014045a5e76 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:42.778695 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991c38a-aa56-4ee0-8c46-5014045a5e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:42.796995 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-3991c38a-aa56-4ee0-8c46-5014045a5e76 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:42.798762 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-3991c38a-aa56-4ee0-8c46-5014045a5e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:42.879838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1367/2700] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:27:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:27:42.948375 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a4b05-48ce-4b3b-82b4-ecc8f7bebd11 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:42.948668 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a4b05-48ce-4b3b-82b4-ecc8f7bebd11 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:42.948948 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a4b05-48ce-4b3b-82b4-ecc8f7bebd11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:42.968541 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-5a5a4b05-48ce-4b3b-82b4-ecc8f7bebd11 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:42.970600 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-5a5a4b05-48ce-4b3b-82b4-ecc8f7bebd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:42.979907 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1334/2701] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:27:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:27:42.983944 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991c38a-aa56-4ee0-8c46-5014045a5e76 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:42.994585 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991c38a-aa56-4ee0-8c46-5014045a5e76 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:43.000738 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ce93c-e855-4586-87df-5f4700e359f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:43.001522 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ce93c-e855-4586-87df-5f4700e359f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:43.001941 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ce93c-e855-4586-87df-5f4700e359f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:43.002425 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ce93c-e855-4586-87df-5f4700e359f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:43.003084 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ce93c-e855-4586-87df-5f4700e359f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:27:43.003396 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ce93c-e855-4586-87df-5f4700e359f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:43.005055 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ce93c-e855-4586-87df-5f4700e359f7 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:43.017329 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1368/2702] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:27:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:27:43.021648 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a4b05-48ce-4b3b-82b4-ecc8f7bebd11 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:43.033265 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a4b05-48ce-4b3b-82b4-ecc8f7bebd11 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:43.038491 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4169b14-62f8-4f37-943b-4e079af6a644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:43.039033 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4169b14-62f8-4f37-943b-4e079af6a644 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:43.039477 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4169b14-62f8-4f37-943b-4e079af6a644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:43.039771 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4169b14-62f8-4f37-943b-4e079af6a644 admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:43.040400 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4169b14-62f8-4f37-943b-4e079af6a644 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:43.042361 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4169b14-62f8-4f37-943b-4e079af6a644 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:43.147377 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1335/2703] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:27:43 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:27:43.148573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ce93c-e855-4586-87df-5f4700e359f7 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:43.159438 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ce93c-e855-4586-87df-5f4700e359f7 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:43.173865 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2838cb6-8579-4335-a555-26b9b12af9df None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:43.174764 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2838cb6-8579-4335-a555-26b9b12af9df None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:43.175218 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2838cb6-8579-4335-a555-26b9b12af9df None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:43.179313 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1336/2704] 10.222.0.98 () {54 vars in 921 bytes} [Thu Aug 18 12:27:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:27:43.189785 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b360e97b-0cbe-426c-ae30-560090eff9ce None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:43.190875 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b360e97b-0cbe-426c-ae30-560090eff9ce None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:43.191477 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b360e97b-0cbe-426c-ae30-560090eff9ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:43.224229 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-b360e97b-0cbe-426c-ae30-560090eff9ce None None] MFA Rules not processed for user `171c72a6b29e418bbb4ea5c6faffe3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:43.226041 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-b360e97b-0cbe-426c-ae30-560090eff9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:43.228390 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49508285-b2f3-4fed-9e9a-f5fd4a8ce568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:43.229515 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49508285-b2f3-4fed-9e9a-f5fd4a8ce568 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:43.229870 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49508285-b2f3-4fed-9e9a-f5fd4a8ce568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:43.230222 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49508285-b2f3-4fed-9e9a-f5fd4a8ce568 admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:43.230989 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49508285-b2f3-4fed-9e9a-f5fd4a8ce568 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:43.236263 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49508285-b2f3-4fed-9e9a-f5fd4a8ce568 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:43.301374 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1337/2705] 10.222.0.98 () {58 vars in 1018 bytes} [Thu Aug 18 12:27:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:27:43.305690 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1369/2706] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:27:43 2022] POST /identity/v3/users => generated 287 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:27:43.309647 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b360e97b-0cbe-426c-ae30-560090eff9ce None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:43.312711 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49508285-b2f3-4fed-9e9a-f5fd4a8ce568 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:43.314573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-49508285-b2f3-4fed-9e9a-f5fd4a8ce568 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:43.320269 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b360e97b-0cbe-426c-ae30-560090eff9ce None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:43.325198 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a98b2-202f-4f07-8ea3-459317686b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:43.325923 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a98b2-202f-4f07-8ea3-459317686b1c admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:43.326306 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a98b2-202f-4f07-8ea3-459317686b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:43.326715 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a98b2-202f-4f07-8ea3-459317686b1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:43.327518 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a98b2-202f-4f07-8ea3-459317686b1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:43.329759 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a98b2-202f-4f07-8ea3-459317686b1c admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:43.340125 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1338/2707] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:27:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:27:43.344085 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a98b2-202f-4f07-8ea3-459317686b1c admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:43.354447 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a98b2-202f-4f07-8ea3-459317686b1c admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:43.359615 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8919b1f8-7cd3-46e1-897f-391df1fc7df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:43.360261 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919b1f8-7cd3-46e1-897f-391df1fc7df8 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:43.360543 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919b1f8-7cd3-46e1-897f-391df1fc7df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:43.360815 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919b1f8-7cd3-46e1-897f-391df1fc7df8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:43.361448 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8919b1f8-7cd3-46e1-897f-391df1fc7df8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:43.363027 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8919b1f8-7cd3-46e1-897f-391df1fc7df8 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:43.377420 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1339/2708] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:27:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 12:27:43.377660 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49508285-b2f3-4fed-9e9a-f5fd4a8ce568 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:43.379141 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8919b1f8-7cd3-46e1-897f-391df1fc7df8 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:43.380175 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f463d9e1-23b2-40bb-9080-58eda065b72e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd020063702b0481e9f4111680766f7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:43.380831 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463d9e1-23b2-40bb-9080-58eda065b72e service glance] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:43.381117 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463d9e1-23b2-40bb-9080-58eda065b72e service glance] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:43.381400 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463d9e1-23b2-40bb-9080-58eda065b72e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:43.389089 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8919b1f8-7cd3-46e1-897f-391df1fc7df8 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:43.392931 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f463d9e1-23b2-40bb-9080-58eda065b72e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:43.394393 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f463d9e1-23b2-40bb-9080-58eda065b72e service glance] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:43.394566 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76923dc1-8db0-4232-94f6-597cc103e02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:43.395158 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76923dc1-8db0-4232-94f6-597cc103e02d admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:43.395458 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76923dc1-8db0-4232-94f6-597cc103e02d admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:43.395731 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76923dc1-8db0-4232-94f6-597cc103e02d admin admin] PATH_INFO: `/v3/projects/e453baf77e6744eb8f001a1c1bb9d0d1/users/e664528e3e33426aae2d47640b57e079/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:43.405984 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1370/2709] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:27:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:27:43.413221 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76923dc1-8db0-4232-94f6-597cc103e02d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e453baf77e6744eb8f001a1c1bb9d0d1, user_id=e664528e3e33426aae2d47640b57e079, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:43.415907 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76923dc1-8db0-4232-94f6-597cc103e02d admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:43.484258 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1340/2710] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:27:43 2022] PUT /identity/v3/projects/e453baf77e6744eb8f001a1c1bb9d0d1/users/e664528e3e33426aae2d47640b57e079/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:27:43.489667 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa8f86-58a5-48d5-85a1-9108c345bed2 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:43.490566 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa8f86-58a5-48d5-85a1-9108c345bed2 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:43.490566 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa8f86-58a5-48d5-85a1-9108c345bed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:43.504710 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:27:43.510799 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-77aa8f86-58a5-48d5-85a1-9108c345bed2 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:43.512160 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-77aa8f86-58a5-48d5-85a1-9108c345bed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:43.573033 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1371/2711] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:27:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:27:43.584209 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76923dc1-8db0-4232-94f6-597cc103e02d admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:43.586305 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-76923dc1-8db0-4232-94f6-597cc103e02d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:43.623880 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:27:43.645862 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76923dc1-8db0-4232-94f6-597cc103e02d admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:43.649463 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5bb52-c27f-4707-afee-56812c1efb5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:43.650059 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5bb52-c27f-4707-afee-56812c1efb5f service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:43.650338 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5bb52-c27f-4707-afee-56812c1efb5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:43.650678 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5bb52-c27f-4707-afee-56812c1efb5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:43.660309 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb5bb52-c27f-4707-afee-56812c1efb5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:43.661648 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb5bb52-c27f-4707-afee-56812c1efb5f service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:43.671607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1341/2712] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:27:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:27:43.731971 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5387f7b4-19de-48cb-88e5-c78b08ee8d0e None None] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:43.732259 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5387f7b4-19de-48cb-88e5-c78b08ee8d0e None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:43.732533 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5387f7b4-19de-48cb-88e5-c78b08ee8d0e None None] PATH_INFO: `/` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:43.733969 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1372/2713] 10.222.0.98 () {54 vars in 921 bytes} [Thu Aug 18 12:27:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:27:43.738616 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a368268-1ac6-41d0-8be5-f98acc0c33f4 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:43.738896 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a368268-1ac6-41d0-8be5-f98acc0c33f4 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:43.739173 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a368268-1ac6-41d0-8be5-f98acc0c33f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:43.759679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-6a368268-1ac6-41d0-8be5-f98acc0c33f4 None None] MFA Rules not processed for user `185644affb284c2b91e604cf99803ffb`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:43.760901 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-6a368268-1ac6-41d0-8be5-f98acc0c33f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:43.768345 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1342/2714] 10.222.0.98 () {58 vars in 1018 bytes} [Thu Aug 18 12:27:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:27:43.780341 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5387f7b4-19de-48cb-88e5-c78b08ee8d0e None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:43.782270 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-5387f7b4-19de-48cb-88e5-c78b08ee8d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:43.797054 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5387f7b4-19de-48cb-88e5-c78b08ee8d0e None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:43.800772 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3b716c-e2fe-4b68-8149-ccaff44174fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:43.801453 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b716c-e2fe-4b68-8149-ccaff44174fd service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:43.801819 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b716c-e2fe-4b68-8149-ccaff44174fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:43.802168 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b716c-e2fe-4b68-8149-ccaff44174fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:43.810483 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3b716c-e2fe-4b68-8149-ccaff44174fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:43.811784 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3b716c-e2fe-4b68-8149-ccaff44174fd service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:43.822303 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1373/2715] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:27:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:27:43.909974 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a6efc-8951-4fd8-bf87-fbbe5f3268c1 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:43.910262 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a6efc-8951-4fd8-bf87-fbbe5f3268c1 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:43.910619 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a6efc-8951-4fd8-bf87-fbbe5f3268c1 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:43.912043 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1343/2716] 10.222.0.98 () {54 vars in 921 bytes} [Thu Aug 18 12:27:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 18 12:27:43.917057 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e47cc-7464-441a-b2ae-be1e10416c29 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:43.917345 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e47cc-7464-441a-b2ae-be1e10416c29 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:43.917628 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e47cc-7464-441a-b2ae-be1e10416c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:43.937497 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-a51e47cc-7464-441a-b2ae-be1e10416c29 None None] MFA Rules not processed for user `171c72a6b29e418bbb4ea5c6faffe3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:43.938712 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-a51e47cc-7464-441a-b2ae-be1e10416c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:44.013309 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1374/2717] 10.222.0.98 () {58 vars in 1018 bytes} [Thu Aug 18 12:27:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:27:44.024920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2a6efc-8951-4fd8-bf87-fbbe5f3268c1 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:44.035787 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2a6efc-8951-4fd8-bf87-fbbe5f3268c1 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:44.039475 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777b7ab-4e0d-4ed0-859f-4ebf6e67b7b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:44.040088 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777b7ab-4e0d-4ed0-859f-4ebf6e67b7b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:44.040365 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777b7ab-4e0d-4ed0-859f-4ebf6e67b7b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:44.040642 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777b7ab-4e0d-4ed0-859f-4ebf6e67b7b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:44.052091 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e777b7ab-4e0d-4ed0-859f-4ebf6e67b7b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:44.053637 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e777b7ab-4e0d-4ed0-859f-4ebf6e67b7b5 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:44.065638 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1344/2718] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:27:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:27:46.962523 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad96ca-76ac-48f2-bb76-669a1f21ced0 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:46.963487 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad96ca-76ac-48f2-bb76-669a1f21ced0 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:46.963952 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad96ca-76ac-48f2-bb76-669a1f21ced0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:46.985032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-a0ad96ca-76ac-48f2-bb76-669a1f21ced0 None None] MFA Rules not processed for user `b6880278f7304051b9ef03e927637e74`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:46.986368 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-a0ad96ca-76ac-48f2-bb76-669a1f21ced0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:47.217420 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1375/2719] 10.222.0.98 () {58 vars in 1009 bytes} [Thu Aug 18 12:27:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:27:47.230678 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777b7ab-4e0d-4ed0-859f-4ebf6e67b7b5 service neutron] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:47.233025 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-e777b7ab-4e0d-4ed0-859f-4ebf6e67b7b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:47.328494 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777b7ab-4e0d-4ed0-859f-4ebf6e67b7b5 service neutron] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:47.338238 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ddc03-8c43-496e-a718-a6429dee7244 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771591fe31cb4e8c9c4320c2a7b2719d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:47.339264 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ddc03-8c43-496e-a718-a6429dee7244 service swift] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:47.340891 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ddc03-8c43-496e-a718-a6429dee7244 service swift] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:47.340891 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ddc03-8c43-496e-a718-a6429dee7244 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:47.350188 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762ddc03-8c43-496e-a718-a6429dee7244 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:47.352383 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762ddc03-8c43-496e-a718-a6429dee7244 service swift] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:47.363076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1345/2720] 10.222.0.98 () {60 vars in 1384 bytes} [Thu Aug 18 12:27:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 12:27:51.323543 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983685d-6122-49e5-b27d-a3acb615fade None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:51.324468 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983685d-6122-49e5-b27d-a3acb615fade None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:51.324922 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983685d-6122-49e5-b27d-a3acb615fade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:51.380887 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-0983685d-6122-49e5-b27d-a3acb615fade None None] MFA Rules not processed for user `e664528e3e33426aae2d47640b57e079`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:27:51.382374 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-0983685d-6122-49e5-b27d-a3acb615fade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:27:51.409354 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:27:51.471922 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1376/2721] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:27:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:27:51.476465 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ddc03-8c43-496e-a718-a6429dee7244 service swift] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:51.487940 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ddc03-8c43-496e-a718-a6429dee7244 service swift] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:51.495220 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f6fa7-ff0d-45e1-a005-c0b33aee9581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:51.495220 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f6fa7-ff0d-45e1-a005-c0b33aee9581 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:51.495220 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f6fa7-ff0d-45e1-a005-c0b33aee9581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:51.495220 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f6fa7-ff0d-45e1-a005-c0b33aee9581 admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:51.495834 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6f6fa7-ff0d-45e1-a005-c0b33aee9581 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:51.496261 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6f6fa7-ff0d-45e1-a005-c0b33aee9581 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:51.643533 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1346/2722] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:27:51 2022] POST /identity/v3/projects => generated 371 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:27:51.645122 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983685d-6122-49e5-b27d-a3acb615fade None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:51.656644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983685d-6122-49e5-b27d-a3acb615fade None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:51.763318 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efddbbe6-6286-44a6-9d68-dfec0cfde715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:51.764056 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efddbbe6-6286-44a6-9d68-dfec0cfde715 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:51.764523 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efddbbe6-6286-44a6-9d68-dfec0cfde715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:51.764802 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efddbbe6-6286-44a6-9d68-dfec0cfde715 admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:51.772161 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efddbbe6-6286-44a6-9d68-dfec0cfde715 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:51.774577 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efddbbe6-6286-44a6-9d68-dfec0cfde715 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:51.893276 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1377/2723] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:27:51 2022] POST /identity/v3/users => generated 292 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:27:51.893978 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f6fa7-ff0d-45e1-a005-c0b33aee9581 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:51.904790 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f6fa7-ff0d-45e1-a005-c0b33aee9581 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:51.913383 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4243d6-5315-4958-89aa-67973468a570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:51.913383 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4243d6-5315-4958-89aa-67973468a570 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:51.913383 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4243d6-5315-4958-89aa-67973468a570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:51.913741 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4243d6-5315-4958-89aa-67973468a570 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:51.914458 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4243d6-5315-4958-89aa-67973468a570 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:51.918322 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4243d6-5315-4958-89aa-67973468a570 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:51.933740 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1347/2724] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:27:51.935638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efddbbe6-6286-44a6-9d68-dfec0cfde715 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:51.949033 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efddbbe6-6286-44a6-9d68-dfec0cfde715 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:51.955865 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491488a-ee4d-4446-b347-1b4efb382c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:51.956588 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491488a-ee4d-4446-b347-1b4efb382c79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:51.956871 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491488a-ee4d-4446-b347-1b4efb382c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:51.957149 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491488a-ee4d-4446-b347-1b4efb382c79 admin admin] PATH_INFO: `/v3/projects/ac0531b9963648aeae3dc01ee6b37d61/users/25515fa352914c2c939f69e23cae78b4/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:51.976380 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7491488a-ee4d-4446-b347-1b4efb382c79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac0531b9963648aeae3dc01ee6b37d61, user_id=25515fa352914c2c939f69e23cae78b4, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:51.979763 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7491488a-ee4d-4446-b347-1b4efb382c79 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:52.048901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1378/2725] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:27:51 2022] PUT /identity/v3/projects/ac0531b9963648aeae3dc01ee6b37d61/users/25515fa352914c2c939f69e23cae78b4/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:27:52.049858 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4243d6-5315-4958-89aa-67973468a570 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:52.062663 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4243d6-5315-4958-89aa-67973468a570 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:52.143886 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5707f672-2b8c-4180-a98d-3e2d73bb9d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:52.144601 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707f672-2b8c-4180-a98d-3e2d73bb9d56 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:52.144832 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707f672-2b8c-4180-a98d-3e2d73bb9d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:52.145111 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707f672-2b8c-4180-a98d-3e2d73bb9d56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:52.145779 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5707f672-2b8c-4180-a98d-3e2d73bb9d56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:52.147466 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5707f672-2b8c-4180-a98d-3e2d73bb9d56 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:52.167219 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1348/2726] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 12:27:52.167717 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491488a-ee4d-4446-b347-1b4efb382c79 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:27:52.179759 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491488a-ee4d-4446-b347-1b4efb382c79 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:27:52.186661 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84fbde-4472-4bb3-84e9-2cac6a0ab912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:27:52.187351 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84fbde-4472-4bb3-84e9-2cac6a0ab912 admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:52.187693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84fbde-4472-4bb3-84e9-2cac6a0ab912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:52.187975 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84fbde-4472-4bb3-84e9-2cac6a0ab912 admin admin] PATH_INFO: `/v3/projects/ac0531b9963648aeae3dc01ee6b37d61/users/25515fa352914c2c939f69e23cae78b4/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:27:52.190608 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f84fbde-4472-4bb3-84e9-2cac6a0ab912 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac0531b9963648aeae3dc01ee6b37d61, user_id=25515fa352914c2c939f69e23cae78b4, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:27:52.193351 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f84fbde-4472-4bb3-84e9-2cac6a0ab912 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:27:52.214031 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1379/2727] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:27:52 2022] PUT /identity/v3/projects/ac0531b9963648aeae3dc01ee6b37d61/users/25515fa352914c2c939f69e23cae78b4/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:27:59.992494 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736b0a6-a680-4c07-9572-bf9cc92ce11b None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:27:59.996280 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736b0a6-a680-4c07-9572-bf9cc92ce11b None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:27:59.996280 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736b0a6-a680-4c07-9572-bf9cc92ce11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:00.014096 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:28:00.034336 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-3736b0a6-a680-4c07-9572-bf9cc92ce11b None None] MFA Rules not processed for user `25515fa352914c2c939f69e23cae78b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:28:00.034336 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-3736b0a6-a680-4c07-9572-bf9cc92ce11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:28:00.117987 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1349/2728] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:27:59 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:28:00.134290 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84fbde-4472-4bb3-84e9-2cac6a0ab912 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:28:00.136111 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-6f84fbde-4472-4bb3-84e9-2cac6a0ab912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:28:00.220311 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84fbde-4472-4bb3-84e9-2cac6a0ab912 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:28:00.225921 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf19e4d-6eb6-4189-814e-0aee33fbe30d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:28:00.226540 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf19e4d-6eb6-4189-814e-0aee33fbe30d service nova] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:00.226827 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf19e4d-6eb6-4189-814e-0aee33fbe30d service nova] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:00.227106 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf19e4d-6eb6-4189-814e-0aee33fbe30d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:00.242740 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:28:00.292340 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf19e4d-6eb6-4189-814e-0aee33fbe30d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:28:00.293998 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf19e4d-6eb6-4189-814e-0aee33fbe30d service nova] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:28:00.304167 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1380/2729] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:28:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:28:04.113848 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d267212-8950-41d8-842f-bf55ecb836b0 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:04.114349 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d267212-8950-41d8-842f-bf55ecb836b0 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:04.114431 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d267212-8950-41d8-842f-bf55ecb836b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:04.135377 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-1d267212-8950-41d8-842f-bf55ecb836b0 None None] MFA Rules not processed for user `b6880278f7304051b9ef03e927637e74`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:28:04.138415 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-1d267212-8950-41d8-842f-bf55ecb836b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:28:04.223575 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1350/2730] 10.222.0.98 () {58 vars in 967 bytes} [Thu Aug 18 12:28:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:28:04.234347 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf19e4d-6eb6-4189-814e-0aee33fbe30d service nova] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:28:04.294548 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ebf28-ccb5-465a-a389-ac6e3ebfa211 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:04.294862 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ebf28-ccb5-465a-a389-ac6e3ebfa211 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:04.295089 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ebf28-ccb5-465a-a389-ac6e3ebfa211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:04.324165 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-993ebf28-ccb5-465a-a389-ac6e3ebfa211 None None] MFA Rules not processed for user `e664528e3e33426aae2d47640b57e079`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:28:04.324165 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-993ebf28-ccb5-465a-a389-ac6e3ebfa211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:28:04.326972 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf19e4d-6eb6-4189-814e-0aee33fbe30d service nova] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:28:04.330439 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1351/2731] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:28:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:28:04.336622 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aea835-b64a-4f14-aef8-ef90d7999b80 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771591fe31cb4e8c9c4320c2a7b2719d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:28:04.337738 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aea835-b64a-4f14-aef8-ef90d7999b80 service swift] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:04.338249 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aea835-b64a-4f14-aef8-ef90d7999b80 service swift] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:04.338897 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aea835-b64a-4f14-aef8-ef90d7999b80 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:04.352731 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aea835-b64a-4f14-aef8-ef90d7999b80 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:28:04.354723 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aea835-b64a-4f14-aef8-ef90d7999b80 service swift] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:28:04.369105 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1381/2732] 10.222.0.98 () {60 vars in 1384 bytes} [Thu Aug 18 12:28:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 12:28:09.679162 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24e1ba-9a38-47f0-97af-0d270076fc43 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:09.679699 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24e1ba-9a38-47f0-97af-0d270076fc43 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:09.679843 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24e1ba-9a38-47f0-97af-0d270076fc43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:09.700076 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-3c24e1ba-9a38-47f0-97af-0d270076fc43 None None] MFA Rules not processed for user `b6880278f7304051b9ef03e927637e74`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:28:09.701065 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-3c24e1ba-9a38-47f0-97af-0d270076fc43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:28:09.708936 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1352/2733] 10.222.0.98 () {58 vars in 1009 bytes} [Thu Aug 18 12:28:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:28:09.721551 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aea835-b64a-4f14-aef8-ef90d7999b80 service swift] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:28:09.732522 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aea835-b64a-4f14-aef8-ef90d7999b80 service swift] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:28:09.738334 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f360a7f7-a5ae-401e-a517-3ed288cf838a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771591fe31cb4e8c9c4320c2a7b2719d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:28:09.738939 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360a7f7-a5ae-401e-a517-3ed288cf838a service swift] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:09.739228 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360a7f7-a5ae-401e-a517-3ed288cf838a service swift] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:09.739533 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360a7f7-a5ae-401e-a517-3ed288cf838a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:09.748774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f360a7f7-a5ae-401e-a517-3ed288cf838a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:28:09.750343 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f360a7f7-a5ae-401e-a517-3ed288cf838a service swift] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:28:09.758526 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1382/2734] 10.222.0.98 () {60 vars in 1384 bytes} [Thu Aug 18 12:28:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 12:28:19.693977 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=61 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:28:19.755242 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=61 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:28:30.649575 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24e1ba-9a38-47f0-97af-0d270076fc43 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:28:30.651030 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-3c24e1ba-9a38-47f0-97af-0d270076fc43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:28:30.809499 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0079730-5c50-4e5c-bba0-20149c2f1e29 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:30.810713 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0079730-5c50-4e5c-bba0-20149c2f1e29 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:30.811239 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0079730-5c50-4e5c-bba0-20149c2f1e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:30.834052 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24e1ba-9a38-47f0-97af-0d270076fc43 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:28:30.839687 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270e79a7-9e40-4d8a-978d-b4a488ce10e4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193637acadc14ec4a48ab79566ba3dab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:28:30.840503 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e79a7-9e40-4d8a-978d-b4a488ce10e4 service placement] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:30.840794 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e79a7-9e40-4d8a-978d-b4a488ce10e4 service placement] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:30.841069 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e79a7-9e40-4d8a-978d-b4a488ce10e4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:30.842925 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-270e79a7-9e40-4d8a-978d-b4a488ce10e4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:28:30.846149 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-f0079730-5c50-4e5c-bba0-20149c2f1e29 None None] MFA Rules not processed for user `b6880278f7304051b9ef03e927637e74`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:28:30.847403 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-f0079730-5c50-4e5c-bba0-20149c2f1e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:28:30.858851 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270e79a7-9e40-4d8a-978d-b4a488ce10e4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:28:30.861051 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270e79a7-9e40-4d8a-978d-b4a488ce10e4 service placement] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:28:30.864520 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1383/2735] 10.222.0.98 () {58 vars in 967 bytes} [Thu Aug 18 12:28:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:28:30.871881 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1353/2736] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:28:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:28:30.879359 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0079730-5c50-4e5c-bba0-20149c2f1e29 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:28:30.893543 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0079730-5c50-4e5c-bba0-20149c2f1e29 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:28:30.898772 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd87b63c-a449-4f50-b97c-a7bee083a4c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '771591fe31cb4e8c9c4320c2a7b2719d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:28:30.899389 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87b63c-a449-4f50-b97c-a7bee083a4c5 service swift] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:30.899709 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87b63c-a449-4f50-b97c-a7bee083a4c5 service swift] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:30.899988 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87b63c-a449-4f50-b97c-a7bee083a4c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:30.909618 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd87b63c-a449-4f50-b97c-a7bee083a4c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:28:30.911290 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd87b63c-a449-4f50-b97c-a7bee083a4c5 service swift] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:28:30.920262 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1384/2737] 10.222.0.98 () {60 vars in 1384 bytes} [Thu Aug 18 12:28:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 18 12:28:33.869993 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270e79a7-9e40-4d8a-978d-b4a488ce10e4 service placement] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:28:33.873022 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-270e79a7-9e40-4d8a-978d-b4a488ce10e4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:28:33.889567 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270e79a7-9e40-4d8a-978d-b4a488ce10e4 service placement] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:28:33.895377 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e377c-f385-49c1-b442-7e55aa5e05c2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193637acadc14ec4a48ab79566ba3dab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:28:33.896023 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e377c-f385-49c1-b442-7e55aa5e05c2 service placement] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:33.896308 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e377c-f385-49c1-b442-7e55aa5e05c2 service placement] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:33.896584 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e377c-f385-49c1-b442-7e55aa5e05c2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:33.898561 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-555e377c-f385-49c1-b442-7e55aa5e05c2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:28:33.915308 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e377c-f385-49c1-b442-7e55aa5e05c2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:28:33.917078 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e377c-f385-49c1-b442-7e55aa5e05c2 service placement] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:28:33.930750 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1354/2738] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:28:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:28:33.991773 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd87b63c-a449-4f50-b97c-a7bee083a4c5 service swift] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:28:33.994151 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-fd87b63c-a449-4f50-b97c-a7bee083a4c5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:28:34.104442 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd87b63c-a449-4f50-b97c-a7bee083a4c5 service swift] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:28:34.110418 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70686b1-454d-4447-b36c-197238ccd680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:28:34.111102 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70686b1-454d-4447-b36c-197238ccd680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:34.111389 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70686b1-454d-4447-b36c-197238ccd680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:34.111691 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70686b1-454d-4447-b36c-197238ccd680 admin admin] PATH_INFO: `/v3/users/cb91581b839c4318a1bb6c53e63f2649` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:34.112919 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70686b1-454d-4447-b36c-197238ccd680 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb91581b839c4318a1bb6c53e63f2649)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:28:34.115100 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70686b1-454d-4447-b36c-197238ccd680 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:28:34.349203 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-b70686b1-454d-4447-b36c-197238ccd680 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb91581b839c4318a1bb6c53e63f2649'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:28:34.359138 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-b70686b1-454d-4447-b36c-197238ccd680 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb91581b839c4318a1bb6c53e63f2649'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:28:34.360924 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-b70686b1-454d-4447-b36c-197238ccd680 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb91581b839c4318a1bb6c53e63f2649'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:28:34.377796 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-b70686b1-454d-4447-b36c-197238ccd680 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb91581b839c4318a1bb6c53e63f2649'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:28:34.473779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-b70686b1-454d-4447-b36c-197238ccd680 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb91581b839c4318a1bb6c53e63f2649'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:28:34.481125 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1385/2739] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:28:33 2022] DELETE /identity/v3/users/cb91581b839c4318a1bb6c53e63f2649 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:28:34.746123 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e377c-f385-49c1-b442-7e55aa5e05c2 service placement] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:28:34.756783 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e377c-f385-49c1-b442-7e55aa5e05c2 service placement] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:28:34.821576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cbb8a0-7ca0-451b-8d83-8e1efddab358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:28:34.822250 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cbb8a0-7ca0-451b-8d83-8e1efddab358 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:34.822600 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cbb8a0-7ca0-451b-8d83-8e1efddab358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:34.822929 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cbb8a0-7ca0-451b-8d83-8e1efddab358 admin admin] PATH_INFO: `/v3/projects/672f308b487a4c0bb2d3e731542289cf` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:34.824127 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cbb8a0-7ca0-451b-8d83-8e1efddab358 admin admin] RBAC: Authorizing `identity:delete_project(project_id=672f308b487a4c0bb2d3e731542289cf)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:28:34.826098 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cbb8a0-7ca0-451b-8d83-8e1efddab358 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:28:34.837353 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:28:34.948944 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-49cbb8a0-7ca0-451b-8d83-8e1efddab358 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '672f308b487a4c0bb2d3e731542289cf', 'request_id': 'req-49cbb8a0-7ca0-451b-8d83-8e1efddab358'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:28:34.954911 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-49cbb8a0-7ca0-451b-8d83-8e1efddab358 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '672f308b487a4c0bb2d3e731542289cf', 'request_id': 'req-49cbb8a0-7ca0-451b-8d83-8e1efddab358'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:28:35.011107 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-49cbb8a0-7ca0-451b-8d83-8e1efddab358 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '672f308b487a4c0bb2d3e731542289cf', 'request_id': 'req-49cbb8a0-7ca0-451b-8d83-8e1efddab358'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:28:35.032505 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-49cbb8a0-7ca0-451b-8d83-8e1efddab358 admin admin] The token cache is being invalidate because project 672f308b487a4c0bb2d3e731542289cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:28:35.032505 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-49cbb8a0-7ca0-451b-8d83-8e1efddab358 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:28:35.032505 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-49cbb8a0-7ca0-451b-8d83-8e1efddab358 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:28:35.032505 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1355/2740] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:28:34 2022] DELETE /identity/v3/projects/672f308b487a4c0bb2d3e731542289cf => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:28:35.033782 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292aac0-2326-410d-a4f2-a70408938dea None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:35.034293 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292aac0-2326-410d-a4f2-a70408938dea None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:35.034293 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292aac0-2326-410d-a4f2-a70408938dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:35.054583 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:28:35.091566 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-b292aac0-2326-410d-a4f2-a70408938dea None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:28:35.092776 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-b292aac0-2326-410d-a4f2-a70408938dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:28:35.152224 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1386/2741] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:28:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:28:35.160387 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9137e9c2-14b6-4ffb-9d9c-2f1e1101a5bb None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:35.160684 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9137e9c2-14b6-4ffb-9d9c-2f1e1101a5bb None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:35.160966 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9137e9c2-14b6-4ffb-9d9c-2f1e1101a5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:35.180075 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-9137e9c2-14b6-4ffb-9d9c-2f1e1101a5bb None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:28:35.181287 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-9137e9c2-14b6-4ffb-9d9c-2f1e1101a5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:28:35.189200 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1356/2742] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:28:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:28:35.193531 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b292aac0-2326-410d-a4f2-a70408938dea None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:28:35.203926 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b292aac0-2326-410d-a4f2-a70408938dea None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:28:35.208448 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49f472-108b-4e45-92f3-839096e21ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:28:35.209040 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49f472-108b-4e45-92f3-839096e21ec9 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:35.209317 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49f472-108b-4e45-92f3-839096e21ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:35.209585 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49f472-108b-4e45-92f3-839096e21ec9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:35.210228 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d49f472-108b-4e45-92f3-839096e21ec9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:28:35.210637 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d49f472-108b-4e45-92f3-839096e21ec9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:28:35.212254 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d49f472-108b-4e45-92f3-839096e21ec9 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:28:35.224275 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1387/2743] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:28:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:28:35.228771 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9137e9c2-14b6-4ffb-9d9c-2f1e1101a5bb None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:28:35.238619 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9137e9c2-14b6-4ffb-9d9c-2f1e1101a5bb None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:28:35.243426 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013279bc-70d6-4910-80d4-c6efe80c52bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:28:35.244056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013279bc-70d6-4910-80d4-c6efe80c52bd admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:35.244336 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013279bc-70d6-4910-80d4-c6efe80c52bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:35.244611 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013279bc-70d6-4910-80d4-c6efe80c52bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:35.245290 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013279bc-70d6-4910-80d4-c6efe80c52bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:28:35.247181 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013279bc-70d6-4910-80d4-c6efe80c52bd admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:28:35.338524 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1357/2744] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:28:35 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:28:35.342749 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49f472-108b-4e45-92f3-839096e21ec9 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:28:35.353858 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49f472-108b-4e45-92f3-839096e21ec9 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:28:35.410651 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9185a8b9-35de-404b-afaf-b6861c30e10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:28:35.411260 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9185a8b9-35de-404b-afaf-b6861c30e10b admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:35.411577 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9185a8b9-35de-404b-afaf-b6861c30e10b admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:35.411858 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9185a8b9-35de-404b-afaf-b6861c30e10b admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:35.412498 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9185a8b9-35de-404b-afaf-b6861c30e10b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:28:35.414424 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9185a8b9-35de-404b-afaf-b6861c30e10b admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:28:35.498724 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1388/2745] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:28:35 2022] POST /identity/v3/users => generated 289 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:28:35.503431 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013279bc-70d6-4910-80d4-c6efe80c52bd admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:28:35.514526 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013279bc-70d6-4910-80d4-c6efe80c52bd admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:28:35.520259 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8255042-d41a-431b-9742-039cc48ce228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:28:35.520968 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8255042-d41a-431b-9742-039cc48ce228 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:35.525034 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8255042-d41a-431b-9742-039cc48ce228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:35.525034 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8255042-d41a-431b-9742-039cc48ce228 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:35.525034 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8255042-d41a-431b-9742-039cc48ce228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:28:35.525034 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8255042-d41a-431b-9742-039cc48ce228 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:28:35.534694 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1358/2746] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:28:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:28:35.538993 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9185a8b9-35de-404b-afaf-b6861c30e10b admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:28:35.549708 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9185a8b9-35de-404b-afaf-b6861c30e10b admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:28:35.554565 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b76db9-57ce-4d9c-9380-7a245bc919a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:28:35.555177 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b76db9-57ce-4d9c-9380-7a245bc919a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:35.555493 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b76db9-57ce-4d9c-9380-7a245bc919a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:35.555775 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b76db9-57ce-4d9c-9380-7a245bc919a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:35.556452 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b76db9-57ce-4d9c-9380-7a245bc919a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:28:35.558051 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b76db9-57ce-4d9c-9380-7a245bc919a8 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:28:35.569668 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1389/2747] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:28:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:28:35.574084 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8255042-d41a-431b-9742-039cc48ce228 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:28:35.584971 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8255042-d41a-431b-9742-039cc48ce228 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:28:35.590043 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf058818-02da-49da-b40a-8dedba4a40e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:28:35.590688 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf058818-02da-49da-b40a-8dedba4a40e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:35.590971 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf058818-02da-49da-b40a-8dedba4a40e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:35.591249 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf058818-02da-49da-b40a-8dedba4a40e2 admin admin] PATH_INFO: `/v3/projects/c1534d6a880c4f44b741ff5b208663bc/users/3907830851364a73b4d3715280a132bc/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:35.606108 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf058818-02da-49da-b40a-8dedba4a40e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1534d6a880c4f44b741ff5b208663bc, user_id=3907830851364a73b4d3715280a132bc, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:28:35.608789 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf058818-02da-49da-b40a-8dedba4a40e2 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:28:35.716426 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1359/2748] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:28:35 2022] PUT /identity/v3/projects/c1534d6a880c4f44b741ff5b208663bc/users/3907830851364a73b4d3715280a132bc/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:28:35.721514 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d2229-892f-452f-9d30-45159ee676fb None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:35.721803 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d2229-892f-452f-9d30-45159ee676fb None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:35.722079 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d2229-892f-452f-9d30-45159ee676fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:35.741800 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-e25d2229-892f-452f-9d30-45159ee676fb None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:28:35.743035 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-e25d2229-892f-452f-9d30-45159ee676fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:28:35.790560 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:28:35.805954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1390/2749] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:28:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:28:35.816245 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf058818-02da-49da-b40a-8dedba4a40e2 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:28:35.818224 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-bf058818-02da-49da-b40a-8dedba4a40e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:28:35.880122 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf058818-02da-49da-b40a-8dedba4a40e2 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:28:35.883912 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cdc106-d3b6-4fdc-9bc4-bd2be3437768 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:28:35.884524 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdc106-d3b6-4fdc-9bc4-bd2be3437768 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:35.884813 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdc106-d3b6-4fdc-9bc4-bd2be3437768 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:35.885091 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdc106-d3b6-4fdc-9bc4-bd2be3437768 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:35.894453 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cdc106-d3b6-4fdc-9bc4-bd2be3437768 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:28:35.895876 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cdc106-d3b6-4fdc-9bc4-bd2be3437768 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:28:35.905528 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1360/2750] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:28:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:28:41.446496 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb16b0-a111-4e7f-9327-2bf4005d46d6 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:41.446969 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb16b0-a111-4e7f-9327-2bf4005d46d6 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:41.447081 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb16b0-a111-4e7f-9327-2bf4005d46d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:41.481361 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-2fbb16b0-a111-4e7f-9327-2bf4005d46d6 None None] MFA Rules not processed for user `3907830851364a73b4d3715280a132bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:28:41.482806 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-2fbb16b0-a111-4e7f-9327-2bf4005d46d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:28:41.590612 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1391/2751] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:28:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:28:41.595238 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cdc106-d3b6-4fdc-9bc4-bd2be3437768 service neutron] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:28:41.606338 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cdc106-d3b6-4fdc-9bc4-bd2be3437768 service neutron] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:28:41.611920 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea242a36-3bc6-4f3a-93bb-a9badd9d5154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:28:41.612531 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea242a36-3bc6-4f3a-93bb-a9badd9d5154 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:41.612815 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea242a36-3bc6-4f3a-93bb-a9badd9d5154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:41.613090 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea242a36-3bc6-4f3a-93bb-a9badd9d5154 admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:41.613728 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea242a36-3bc6-4f3a-93bb-a9badd9d5154 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:28:41.615684 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea242a36-3bc6-4f3a-93bb-a9badd9d5154 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:28:41.732082 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1361/2752] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:28:41 2022] POST /identity/v3/projects => generated 377 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:28:41.733425 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb16b0-a111-4e7f-9327-2bf4005d46d6 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:28:41.746414 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb16b0-a111-4e7f-9327-2bf4005d46d6 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:28:41.821998 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1817f49-3626-42ee-ab61-872aa4fa3631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:28:41.822791 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1817f49-3626-42ee-ab61-872aa4fa3631 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:41.823078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1817f49-3626-42ee-ab61-872aa4fa3631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:41.823364 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1817f49-3626-42ee-ab61-872aa4fa3631 admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:41.824052 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1817f49-3626-42ee-ab61-872aa4fa3631 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:28:41.826282 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1817f49-3626-42ee-ab61-872aa4fa3631 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:28:41.921134 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1392/2753] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:28:41 2022] POST /identity/v3/users => generated 295 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:28:41.925434 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea242a36-3bc6-4f3a-93bb-a9badd9d5154 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:28:41.935664 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea242a36-3bc6-4f3a-93bb-a9badd9d5154 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:28:41.942904 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c970bf5-3cfd-4672-83f7-d056f800112b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:28:41.943525 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c970bf5-3cfd-4672-83f7-d056f800112b admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:41.943809 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c970bf5-3cfd-4672-83f7-d056f800112b admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:41.944083 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c970bf5-3cfd-4672-83f7-d056f800112b admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:41.944728 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c970bf5-3cfd-4672-83f7-d056f800112b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:28:41.946314 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c970bf5-3cfd-4672-83f7-d056f800112b admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:28:41.957717 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1362/2754] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:28:41.961992 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1817f49-3626-42ee-ab61-872aa4fa3631 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:28:41.972425 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1817f49-3626-42ee-ab61-872aa4fa3631 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:28:41.977741 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1298145-8a05-41dd-b4b7-b6997e138f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:28:41.978503 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1298145-8a05-41dd-b4b7-b6997e138f0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:41.978802 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1298145-8a05-41dd-b4b7-b6997e138f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:41.979078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1298145-8a05-41dd-b4b7-b6997e138f0b admin admin] PATH_INFO: `/v3/projects/4a0434ddfbda4102a6217e5af1b07d17/users/9fdeb664774a4c6e88ad61d3488960c0/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:41.994935 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1298145-8a05-41dd-b4b7-b6997e138f0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a0434ddfbda4102a6217e5af1b07d17, user_id=9fdeb664774a4c6e88ad61d3488960c0, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:28:41.997617 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1298145-8a05-41dd-b4b7-b6997e138f0b admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:28:42.084550 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1393/2755] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:28:41 2022] PUT /identity/v3/projects/4a0434ddfbda4102a6217e5af1b07d17/users/9fdeb664774a4c6e88ad61d3488960c0/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 18 12:28:42.085779 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c970bf5-3cfd-4672-83f7-d056f800112b admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:28:42.093205 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:28:42.097265 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c970bf5-3cfd-4672-83f7-d056f800112b admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:28:42.154939 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0c6ab-52dc-4019-835f-6558f561d14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:28:42.155583 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0c6ab-52dc-4019-835f-6558f561d14f admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:42.155862 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0c6ab-52dc-4019-835f-6558f561d14f admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:42.156632 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0c6ab-52dc-4019-835f-6558f561d14f admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:42.157278 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a0c6ab-52dc-4019-835f-6558f561d14f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:28:42.159136 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a0c6ab-52dc-4019-835f-6558f561d14f admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:28:42.169131 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1363/2756] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:28:42.174176 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1298145-8a05-41dd-b4b7-b6997e138f0b admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:28:42.185390 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1298145-8a05-41dd-b4b7-b6997e138f0b admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:28:42.190686 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8ae75-74c5-47ca-a124-f8b594c6ee9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:28:42.191171 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8ae75-74c5-47ca-a124-f8b594c6ee9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:42.191492 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8ae75-74c5-47ca-a124-f8b594c6ee9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:42.191780 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8ae75-74c5-47ca-a124-f8b594c6ee9d admin admin] PATH_INFO: `/v3/projects/4a0434ddfbda4102a6217e5af1b07d17/users/9fdeb664774a4c6e88ad61d3488960c0/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:42.193986 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab8ae75-74c5-47ca-a124-f8b594c6ee9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a0434ddfbda4102a6217e5af1b07d17, user_id=9fdeb664774a4c6e88ad61d3488960c0, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:28:42.196613 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab8ae75-74c5-47ca-a124-f8b594c6ee9d admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:28:42.211774 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1394/2757] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:28:42 2022] PUT /identity/v3/projects/4a0434ddfbda4102a6217e5af1b07d17/users/9fdeb664774a4c6e88ad61d3488960c0/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:28:48.133198 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0c6ab-52dc-4019-835f-6558f561d14f admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:28:48.262821 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0c6ab-52dc-4019-835f-6558f561d14f admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:28:48.373629 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9417602-f01b-4a87-aae2-3930778c44ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:28:48.374226 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9417602-f01b-4a87-aae2-3930778c44ba service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:48.375085 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9417602-f01b-4a87-aae2-3930778c44ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:48.375552 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9417602-f01b-4a87-aae2-3930778c44ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:48.378022 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-e9417602-f01b-4a87-aae2-3930778c44ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:28:48.469523 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9417602-f01b-4a87-aae2-3930778c44ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:28:48.471283 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9417602-f01b-4a87-aae2-3930778c44ba service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:28:48.476171 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:28:48.482783 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1364/2758] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:28:48 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:28:49.676088 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a388251-bc89-457f-b954-875801b688d2 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:49.676623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a388251-bc89-457f-b954-875801b688d2 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:49.676623 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a388251-bc89-457f-b954-875801b688d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:49.715900 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-3a388251-bc89-457f-b954-875801b688d2 None None] MFA Rules not processed for user `9fdeb664774a4c6e88ad61d3488960c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:28:49.716657 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-3a388251-bc89-457f-b954-875801b688d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:28:49.740138 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:28:49.801367 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1395/2759] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:28:49 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:28:49.815491 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9417602-f01b-4a87-aae2-3930778c44ba service neutron] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:28:49.817985 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-e9417602-f01b-4a87-aae2-3930778c44ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:28:49.942239 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9417602-f01b-4a87-aae2-3930778c44ba service neutron] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:28:49.947918 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25f19b-f931-48be-b4db-2254354cd467 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:28:49.948496 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25f19b-f931-48be-b4db-2254354cd467 service nova] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:49.955795 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25f19b-f931-48be-b4db-2254354cd467 service nova] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:49.955795 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25f19b-f931-48be-b4db-2254354cd467 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:50.008552 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c25f19b-f931-48be-b4db-2254354cd467 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:28:50.009871 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c25f19b-f931-48be-b4db-2254354cd467 service nova] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:28:50.023204 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1365/2760] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:28:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:28:50.894927 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a388251-bc89-457f-b954-875801b688d2 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:28:50.894927 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-3a388251-bc89-457f-b954-875801b688d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:28:50.931109 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a388251-bc89-457f-b954-875801b688d2 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:28:50.935927 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c4277-c4f0-4b38-82c8-7a07a2bf039a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:28:50.938001 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c4277-c4f0-4b38-82c8-7a07a2bf039a service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:50.938445 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c4277-c4f0-4b38-82c8-7a07a2bf039a service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:50.938881 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c4277-c4f0-4b38-82c8-7a07a2bf039a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:50.941589 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-247c4277-c4f0-4b38-82c8-7a07a2bf039a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:28:51.036379 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247c4277-c4f0-4b38-82c8-7a07a2bf039a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:28:51.037975 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247c4277-c4f0-4b38-82c8-7a07a2bf039a service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:28:51.078788 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1396/2761] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:28:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:28:55.163797 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25f19b-f931-48be-b4db-2254354cd467 service nova] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:28:55.167002 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-8c25f19b-f931-48be-b4db-2254354cd467 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:28:55.185377 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25f19b-f931-48be-b4db-2254354cd467 service nova] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:28:55.190977 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a2bae-3bba-44bc-bb00-17ede9adc28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:28:55.192332 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a2bae-3bba-44bc-bb00-17ede9adc28e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:55.192624 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a2bae-3bba-44bc-bb00-17ede9adc28e admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:55.192988 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a2bae-3bba-44bc-bb00-17ede9adc28e admin admin] PATH_INFO: `/v3/users/d98949bd624a4330a473ebe5de31586a` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:55.194140 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8a2bae-3bba-44bc-bb00-17ede9adc28e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d98949bd624a4330a473ebe5de31586a)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:28:55.196223 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8a2bae-3bba-44bc-bb00-17ede9adc28e admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:28:55.389273 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-bb8a2bae-3bba-44bc-bb00-17ede9adc28e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd98949bd624a4330a473ebe5de31586a'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:28:55.390452 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-bb8a2bae-3bba-44bc-bb00-17ede9adc28e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd98949bd624a4330a473ebe5de31586a'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:28:55.391083 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-bb8a2bae-3bba-44bc-bb00-17ede9adc28e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd98949bd624a4330a473ebe5de31586a'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:28:55.398536 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-bb8a2bae-3bba-44bc-bb00-17ede9adc28e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd98949bd624a4330a473ebe5de31586a'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:28:55.409773 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:28:55.417272 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-bb8a2bae-3bba-44bc-bb00-17ede9adc28e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd98949bd624a4330a473ebe5de31586a'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:28:55.515300 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1366/2762] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:28:55 2022] DELETE /identity/v3/users/d98949bd624a4330a473ebe5de31586a => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:28:55.768546 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c4277-c4f0-4b38-82c8-7a07a2bf039a service neutron] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:28:55.779016 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c4277-c4f0-4b38-82c8-7a07a2bf039a service neutron] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:28:55.842925 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0bb5d4-ef25-4c03-9509-800f094b52f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:28:55.843641 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0bb5d4-ef25-4c03-9509-800f094b52f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:55.843877 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0bb5d4-ef25-4c03-9509-800f094b52f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:55.844218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0bb5d4-ef25-4c03-9509-800f094b52f3 admin admin] PATH_INFO: `/v3/projects/80ccf2ca9a634becb2eee93fddbd2258` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:55.845314 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0bb5d4-ef25-4c03-9509-800f094b52f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80ccf2ca9a634becb2eee93fddbd2258)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:28:55.847237 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0bb5d4-ef25-4c03-9509-800f094b52f3 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:28:55.958815 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:28:55.962521 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-4b0bb5d4-ef25-4c03-9509-800f094b52f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80ccf2ca9a634becb2eee93fddbd2258', 'request_id': 'req-4b0bb5d4-ef25-4c03-9509-800f094b52f3'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:28:55.967921 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-4b0bb5d4-ef25-4c03-9509-800f094b52f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80ccf2ca9a634becb2eee93fddbd2258', 'request_id': 'req-4b0bb5d4-ef25-4c03-9509-800f094b52f3'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:28:56.049935 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-4b0bb5d4-ef25-4c03-9509-800f094b52f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80ccf2ca9a634becb2eee93fddbd2258', 'request_id': 'req-4b0bb5d4-ef25-4c03-9509-800f094b52f3'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:28:56.063857 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-4b0bb5d4-ef25-4c03-9509-800f094b52f3 admin admin] The token cache is being invalidate because project 80ccf2ca9a634becb2eee93fddbd2258 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:28:56.064160 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-4b0bb5d4-ef25-4c03-9509-800f094b52f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:28:56.064843 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-4b0bb5d4-ef25-4c03-9509-800f094b52f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:28:56.066369 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1397/2763] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:28:55 2022] DELETE /identity/v3/projects/80ccf2ca9a634becb2eee93fddbd2258 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:28:56.071534 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b9f1b-6a72-4ec5-b67d-cbebcd72bd63 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:56.071822 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b9f1b-6a72-4ec5-b67d-cbebcd72bd63 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:56.072097 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b9f1b-6a72-4ec5-b67d-cbebcd72bd63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:56.093271 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-329b9f1b-6a72-4ec5-b67d-cbebcd72bd63 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:28:56.094490 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-329b9f1b-6a72-4ec5-b67d-cbebcd72bd63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:28:56.160997 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1367/2764] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:28:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:28:56.172851 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde017e9-d50a-41dd-aedb-daacd43f6373 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:56.173141 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde017e9-d50a-41dd-aedb-daacd43f6373 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:56.173424 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde017e9-d50a-41dd-aedb-daacd43f6373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:56.193569 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-dde017e9-d50a-41dd-aedb-daacd43f6373 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:28:56.194857 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-dde017e9-d50a-41dd-aedb-daacd43f6373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:28:56.203515 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1398/2765] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:28:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:28:56.207944 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b9f1b-6a72-4ec5-b67d-cbebcd72bd63 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:28:56.217699 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b9f1b-6a72-4ec5-b67d-cbebcd72bd63 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:28:56.222437 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f1fc5-9bc4-4aeb-9a7d-d4a72db7671f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:28:56.223044 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f1fc5-9bc4-4aeb-9a7d-d4a72db7671f admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:56.223332 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f1fc5-9bc4-4aeb-9a7d-d4a72db7671f admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:56.223654 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f1fc5-9bc4-4aeb-9a7d-d4a72db7671f admin admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:56.224317 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842f1fc5-9bc4-4aeb-9a7d-d4a72db7671f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:28:56.224624 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842f1fc5-9bc4-4aeb-9a7d-d4a72db7671f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:28:56.226484 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842f1fc5-9bc4-4aeb-9a7d-d4a72db7671f admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:28:56.239240 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1368/2766] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:28:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:28:56.243630 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde017e9-d50a-41dd-aedb-daacd43f6373 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:28:56.253847 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde017e9-d50a-41dd-aedb-daacd43f6373 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:28:56.259179 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dc0401-e676-48ed-81ba-12dc90c842cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:28:56.259837 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc0401-e676-48ed-81ba-12dc90c842cf admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:56.260122 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc0401-e676-48ed-81ba-12dc90c842cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:56.260418 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc0401-e676-48ed-81ba-12dc90c842cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:56.261049 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dc0401-e676-48ed-81ba-12dc90c842cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:28:56.262947 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dc0401-e676-48ed-81ba-12dc90c842cf admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:28:56.368408 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1399/2767] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:28:56 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:28:56.372657 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f1fc5-9bc4-4aeb-9a7d-d4a72db7671f admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:28:56.383717 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f1fc5-9bc4-4aeb-9a7d-d4a72db7671f admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:28:56.443028 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39345a92-9f17-42a6-a770-8dbea44ff6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:28:56.443582 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39345a92-9f17-42a6-a770-8dbea44ff6cb admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:56.443868 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39345a92-9f17-42a6-a770-8dbea44ff6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:56.444140 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39345a92-9f17-42a6-a770-8dbea44ff6cb admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:56.444765 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39345a92-9f17-42a6-a770-8dbea44ff6cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:28:56.446652 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39345a92-9f17-42a6-a770-8dbea44ff6cb admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:28:56.550021 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1369/2768] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:28:56 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:28:56.554372 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dc0401-e676-48ed-81ba-12dc90c842cf admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:28:56.565784 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dc0401-e676-48ed-81ba-12dc90c842cf admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:28:56.570696 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd4a4f-f583-477a-8fe2-fd1fc05be335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:28:56.571310 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd4a4f-f583-477a-8fe2-fd1fc05be335 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:56.571629 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd4a4f-f583-477a-8fe2-fd1fc05be335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:56.575383 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd4a4f-f583-477a-8fe2-fd1fc05be335 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:56.575383 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cd4a4f-f583-477a-8fe2-fd1fc05be335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:28:56.575383 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cd4a4f-f583-477a-8fe2-fd1fc05be335 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:28:56.584654 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1400/2769] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:28:56.588721 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39345a92-9f17-42a6-a770-8dbea44ff6cb admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:28:56.600171 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39345a92-9f17-42a6-a770-8dbea44ff6cb admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:28:56.605418 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cbe92b-950c-4f72-b6b5-14dfc6707de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:28:56.606038 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cbe92b-950c-4f72-b6b5-14dfc6707de9 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:56.606324 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cbe92b-950c-4f72-b6b5-14dfc6707de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:56.606601 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cbe92b-950c-4f72-b6b5-14dfc6707de9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:56.607255 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cbe92b-950c-4f72-b6b5-14dfc6707de9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:28:56.609477 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cbe92b-950c-4f72-b6b5-14dfc6707de9 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:28:56.620561 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1370/2770] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:28:56.624863 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd4a4f-f583-477a-8fe2-fd1fc05be335 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:28:56.635512 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd4a4f-f583-477a-8fe2-fd1fc05be335 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:28:56.641316 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7747f-8336-4cf4-9e01-48d8dfd89a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:28:56.642017 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7747f-8336-4cf4-9e01-48d8dfd89a9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:56.642303 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7747f-8336-4cf4-9e01-48d8dfd89a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:56.642582 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7747f-8336-4cf4-9e01-48d8dfd89a9c admin admin] PATH_INFO: `/v3/projects/86b496e7df3e4e5d8781c49101c619c2/users/27ed3f0ca7c44672a42aa774b02de88c/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:56.658187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f7747f-8336-4cf4-9e01-48d8dfd89a9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=86b496e7df3e4e5d8781c49101c619c2, user_id=27ed3f0ca7c44672a42aa774b02de88c, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:28:56.661640 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f7747f-8336-4cf4-9e01-48d8dfd89a9c admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:28:56.750330 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1401/2771] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:28:56 2022] PUT /identity/v3/projects/86b496e7df3e4e5d8781c49101c619c2/users/27ed3f0ca7c44672a42aa774b02de88c/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:28:56.756791 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8e94e-ecb3-441a-b6c7-f223d95574ac None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:56.757079 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8e94e-ecb3-441a-b6c7-f223d95574ac None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:56.757357 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8e94e-ecb3-441a-b6c7-f223d95574ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:56.776632 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-5ee8e94e-ecb3-441a-b6c7-f223d95574ac None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:28:56.777835 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-5ee8e94e-ecb3-441a-b6c7-f223d95574ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:28:56.826937 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:28:56.839814 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1371/2772] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:28:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:28:56.853967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7747f-8336-4cf4-9e01-48d8dfd89a9c admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:28:56.856303 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-f1f7747f-8336-4cf4-9e01-48d8dfd89a9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:28:56.919457 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7747f-8336-4cf4-9e01-48d8dfd89a9c admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:28:56.923312 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea39efa-e7a4-4a1e-8363-fa4adcc3fb67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:28:56.923957 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea39efa-e7a4-4a1e-8363-fa4adcc3fb67 service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:28:56.924235 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea39efa-e7a4-4a1e-8363-fa4adcc3fb67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:28:56.924506 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea39efa-e7a4-4a1e-8363-fa4adcc3fb67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:28:56.933869 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea39efa-e7a4-4a1e-8363-fa4adcc3fb67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:28:56.935683 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea39efa-e7a4-4a1e-8363-fa4adcc3fb67 service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:28:56.946059 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1402/2773] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:28:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:29:02.411110 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d4ea0-9281-4703-a970-76c7b703a6ad None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:29:02.411672 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d4ea0-9281-4703-a970-76c7b703a6ad None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:29:02.411836 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d4ea0-9281-4703-a970-76c7b703a6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:29:02.448797 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-139d4ea0-9281-4703-a970-76c7b703a6ad None None] MFA Rules not processed for user `27ed3f0ca7c44672a42aa774b02de88c`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:29:02.450524 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-139d4ea0-9281-4703-a970-76c7b703a6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:29:02.528683 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1372/2774] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:29:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:29:02.544125 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea39efa-e7a4-4a1e-8363-fa4adcc3fb67 service neutron] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:29:02.545702 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-5ea39efa-e7a4-4a1e-8363-fa4adcc3fb67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:29:02.635393 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea39efa-e7a4-4a1e-8363-fa4adcc3fb67 service neutron] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:29:02.642139 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0885eea7-bfa0-49af-ae8e-bf68da106b56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:29:02.642833 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0885eea7-bfa0-49af-ae8e-bf68da106b56 service nova] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:29:02.643105 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0885eea7-bfa0-49af-ae8e-bf68da106b56 service nova] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:29:02.643381 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0885eea7-bfa0-49af-ae8e-bf68da106b56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:29:02.654476 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:29:02.654476 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0885eea7-bfa0-49af-ae8e-bf68da106b56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:29:02.654476 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0885eea7-bfa0-49af-ae8e-bf68da106b56 service nova] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:29:02.664422 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1403/2775] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:29:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:29:05.099818 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f2271-edd6-40de-81ff-661f6e2ef919 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:29:05.100305 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f2271-edd6-40de-81ff-661f6e2ef919 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:29:05.101934 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f2271-edd6-40de-81ff-661f6e2ef919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:29:05.121177 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-ce0f2271-edd6-40de-81ff-661f6e2ef919 None None] MFA Rules not processed for user `27ed3f0ca7c44672a42aa774b02de88c`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:29:05.122384 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-ce0f2271-edd6-40de-81ff-661f6e2ef919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:29:05.129883 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1373/2776] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:29:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:29:09.566856 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf5e35-212b-44f9-8f92-7fe8af35a2ad None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:29:09.567346 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf5e35-212b-44f9-8f92-7fe8af35a2ad None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:29:09.567346 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf5e35-212b-44f9-8f92-7fe8af35a2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:29:09.587434 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-1bdf5e35-212b-44f9-8f92-7fe8af35a2ad None None] MFA Rules not processed for user `3907830851364a73b4d3715280a132bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:29:09.588672 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-1bdf5e35-212b-44f9-8f92-7fe8af35a2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:29:09.651278 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1404/2777] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:29:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:29:15.115539 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=38 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:29:19.644102 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=27 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:29:28.899041 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f2271-edd6-40de-81ff-661f6e2ef919 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:29:28.900521 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-ce0f2271-edd6-40de-81ff-661f6e2ef919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:29:28.913334 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f2271-edd6-40de-81ff-661f6e2ef919 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:29:28.916935 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ae036-3678-4dac-8905-13fbd7d09fc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:29:28.917569 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ae036-3678-4dac-8905-13fbd7d09fc3 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:29:28.917852 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ae036-3678-4dac-8905-13fbd7d09fc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:29:28.918129 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ae036-3678-4dac-8905-13fbd7d09fc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:29:28.919884 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-b50ae036-3678-4dac-8905-13fbd7d09fc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:29:28.995197 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ae036-3678-4dac-8905-13fbd7d09fc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:29:28.996591 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ae036-3678-4dac-8905-13fbd7d09fc3 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:29:29.006545 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1374/2778] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:29:28 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:29:39.002044 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=30 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:30:06.216030 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe408b3-430c-49a0-a4dd-2471144acf88 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:30:06.216553 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe408b3-430c-49a0-a4dd-2471144acf88 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:30:06.216553 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe408b3-430c-49a0-a4dd-2471144acf88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:30:06.236776 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-3fe408b3-430c-49a0-a4dd-2471144acf88 None None] MFA Rules not processed for user `27ed3f0ca7c44672a42aa774b02de88c`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:30:06.238429 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-3fe408b3-430c-49a0-a4dd-2471144acf88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:30:06.245455 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1405/2779] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:30:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:30:16.230519 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:30:49.608699 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ae036-3678-4dac-8905-13fbd7d09fc3 service neutron] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:30:49.617666 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ae036-3678-4dac-8905-13fbd7d09fc3 service neutron] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:30:49.621411 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85853136-5919-43af-a78f-1eb2d4b641e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:30:49.622552 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85853136-5919-43af-a78f-1eb2d4b641e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:30:49.622834 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85853136-5919-43af-a78f-1eb2d4b641e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:30:49.623109 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85853136-5919-43af-a78f-1eb2d4b641e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:30:49.624880 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-85853136-5919-43af-a78f-1eb2d4b641e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:30:49.639377 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85853136-5919-43af-a78f-1eb2d4b641e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:30:49.641476 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85853136-5919-43af-a78f-1eb2d4b641e5 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:30:49.653872 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1375/2780] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:30:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:30:50.834393 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe408b3-430c-49a0-a4dd-2471144acf88 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:30:50.844451 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe408b3-430c-49a0-a4dd-2471144acf88 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:30:50.848046 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27af05c5-f04c-4a9d-8e69-2a939b985b5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:30:50.848646 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af05c5-f04c-4a9d-8e69-2a939b985b5b service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:30:50.848935 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af05c5-f04c-4a9d-8e69-2a939b985b5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:30:50.849210 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af05c5-f04c-4a9d-8e69-2a939b985b5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:30:50.850967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-27af05c5-f04c-4a9d-8e69-2a939b985b5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:30:50.868001 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27af05c5-f04c-4a9d-8e69-2a939b985b5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:30:50.869432 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27af05c5-f04c-4a9d-8e69-2a939b985b5b service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:30:50.881458 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1406/2781] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:30:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:30:59.648130 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=6 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:31:00.876121 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=6 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:31:09.086871 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d5621-fca0-41c7-8bdc-2e09af66db8a None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:31:09.087516 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d5621-fca0-41c7-8bdc-2e09af66db8a None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:31:09.087516 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d5621-fca0-41c7-8bdc-2e09af66db8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:31:09.108636 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-3a5d5621-fca0-41c7-8bdc-2e09af66db8a None None] MFA Rules not processed for user `27ed3f0ca7c44672a42aa774b02de88c`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:31:09.109865 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-3a5d5621-fca0-41c7-8bdc-2e09af66db8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:31:09.117655 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1376/2782] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:31:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:31:09.395448 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27af05c5-f04c-4a9d-8e69-2a939b985b5b service neutron] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:31:09.404686 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27af05c5-f04c-4a9d-8e69-2a939b985b5b service neutron] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:31:09.408859 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b041a1-1d76-471d-9de3-7cc4a834fe3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:31:09.409299 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b041a1-1d76-471d-9de3-7cc4a834fe3c service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:31:09.409582 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b041a1-1d76-471d-9de3-7cc4a834fe3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:31:09.409859 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b041a1-1d76-471d-9de3-7cc4a834fe3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:31:09.411686 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-04b041a1-1d76-471d-9de3-7cc4a834fe3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:31:09.422109 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b041a1-1d76-471d-9de3-7cc4a834fe3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:31:09.424212 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b041a1-1d76-471d-9de3-7cc4a834fe3c service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:31:09.433602 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1407/2783] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:31:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:31:09.533191 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d5621-fca0-41c7-8bdc-2e09af66db8a None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:31:09.543354 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d5621-fca0-41c7-8bdc-2e09af66db8a None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:31:09.547011 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26f3493-5623-4f4e-9f62-c25473f75947 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:31:09.548086 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f3493-5623-4f4e-9f62-c25473f75947 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:31:09.548086 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f3493-5623-4f4e-9f62-c25473f75947 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:31:09.548313 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f3493-5623-4f4e-9f62-c25473f75947 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:31:09.550050 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-a26f3493-5623-4f4e-9f62-c25473f75947 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:31:09.564654 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26f3493-5623-4f4e-9f62-c25473f75947 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:31:09.566089 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26f3493-5623-4f4e-9f62-c25473f75947 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:31:09.578981 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1377/2784] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:31:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:31:09.684502 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b041a1-1d76-471d-9de3-7cc4a834fe3c service neutron] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:31:09.687217 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-04b041a1-1d76-471d-9de3-7cc4a834fe3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:31:09.759752 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b041a1-1d76-471d-9de3-7cc4a834fe3c service neutron] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:31:09.764857 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931d1467-5573-4f20-a6f1-dd9106f71f3a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193637acadc14ec4a48ab79566ba3dab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:31:09.765413 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d1467-5573-4f20-a6f1-dd9106f71f3a service placement] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:31:09.765694 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d1467-5573-4f20-a6f1-dd9106f71f3a service placement] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:31:09.765968 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d1467-5573-4f20-a6f1-dd9106f71f3a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:31:09.767727 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-931d1467-5573-4f20-a6f1-dd9106f71f3a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:31:09.779625 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931d1467-5573-4f20-a6f1-dd9106f71f3a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:31:09.781222 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931d1467-5573-4f20-a6f1-dd9106f71f3a service placement] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:31:09.791312 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1408/2785] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:31:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:31:12.061964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26f3493-5623-4f4e-9f62-c25473f75947 service neutron] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:31:12.072254 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26f3493-5623-4f4e-9f62-c25473f75947 service neutron] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:31:12.086210 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddd4399-4c26-45ac-b27c-bfb4eaddd006 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:31:12.092813 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd4399-4c26-45ac-b27c-bfb4eaddd006 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:31:12.092813 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd4399-4c26-45ac-b27c-bfb4eaddd006 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:31:12.092813 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd4399-4c26-45ac-b27c-bfb4eaddd006 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:31:12.092813 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-7ddd4399-4c26-45ac-b27c-bfb4eaddd006 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:31:12.104395 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddd4399-4c26-45ac-b27c-bfb4eaddd006 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:31:12.105976 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddd4399-4c26-45ac-b27c-bfb4eaddd006 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:31:12.116174 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1378/2786] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:31:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:31:19.171246 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931d1467-5573-4f20-a6f1-dd9106f71f3a service placement] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:31:19.183737 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931d1467-5573-4f20-a6f1-dd9106f71f3a service placement] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:31:19.190054 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde621e3-0cc2-465a-bb6d-145c14c8e727 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:31:19.190654 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde621e3-0cc2-465a-bb6d-145c14c8e727 service nova] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:31:19.191024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde621e3-0cc2-465a-bb6d-145c14c8e727 service nova] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:31:19.191304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde621e3-0cc2-465a-bb6d-145c14c8e727 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:31:19.193126 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-bde621e3-0cc2-465a-bb6d-145c14c8e727 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:31:19.206946 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde621e3-0cc2-465a-bb6d-145c14c8e727 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:31:19.208745 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde621e3-0cc2-465a-bb6d-145c14c8e727 service nova] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:31:19.220350 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1409/2787] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:31:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:31:22.111343 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=15 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:31:22.860959 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddd4399-4c26-45ac-b27c-bfb4eaddd006 service neutron] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:31:22.874664 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddd4399-4c26-45ac-b27c-bfb4eaddd006 service neutron] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:31:22.880238 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c9d87-0e41-48c4-82a7-92521c1bcc49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:31:22.880847 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c9d87-0e41-48c4-82a7-92521c1bcc49 service nova] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:31:22.881128 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c9d87-0e41-48c4-82a7-92521c1bcc49 service nova] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:31:22.881458 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c9d87-0e41-48c4-82a7-92521c1bcc49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:31:22.883180 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-3d2c9d87-0e41-48c4-82a7-92521c1bcc49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:31:22.895762 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c9d87-0e41-48c4-82a7-92521c1bcc49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:31:22.897336 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c9d87-0e41-48c4-82a7-92521c1bcc49 service nova] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:31:22.908704 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1379/2788] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:31:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:31:29.215157 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=44 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:31:32.904634 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=6 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:31:33.656373 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7aef1e-a842-4f8d-aaf5-b79ff41fec9e None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:31:33.656871 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7aef1e-a842-4f8d-aaf5-b79ff41fec9e None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:31:33.656985 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7aef1e-a842-4f8d-aaf5-b79ff41fec9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:31:33.677541 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-2d7aef1e-a842-4f8d-aaf5-b79ff41fec9e None None] MFA Rules not processed for user `3907830851364a73b4d3715280a132bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:31:33.678770 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-2d7aef1e-a842-4f8d-aaf5-b79ff41fec9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:31:33.686495 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1410/2789] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:31:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:31:43.670425 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:32:17.072630 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d87995-becb-45b7-9834-39fb2b78bafa None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:32:17.073177 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d87995-becb-45b7-9834-39fb2b78bafa None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:32:17.073177 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d87995-becb-45b7-9834-39fb2b78bafa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:32:17.092686 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-d6d87995-becb-45b7-9834-39fb2b78bafa None None] MFA Rules not processed for user `27ed3f0ca7c44672a42aa774b02de88c`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:32:17.093951 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-d6d87995-becb-45b7-9834-39fb2b78bafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:32:17.100873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1380/2790] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:32:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:32:27.087825 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:32:51.332231 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7aef1e-a842-4f8d-aaf5-b79ff41fec9e None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:32:51.342422 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7aef1e-a842-4f8d-aaf5-b79ff41fec9e None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:32:51.346081 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e1425-70eb-4788-9de2-9a43bd8056d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:32:51.346683 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e1425-70eb-4788-9de2-9a43bd8056d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:32:51.347634 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e1425-70eb-4788-9de2-9a43bd8056d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:32:51.347948 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e1425-70eb-4788-9de2-9a43bd8056d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:32:51.349726 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-537e1425-70eb-4788-9de2-9a43bd8056d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:32:51.362120 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537e1425-70eb-4788-9de2-9a43bd8056d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:32:51.364208 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537e1425-70eb-4788-9de2-9a43bd8056d1 service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:32:51.374106 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1411/2791] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:32:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:32:58.449475 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d87995-becb-45b7-9834-39fb2b78bafa None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:32:58.451090 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-d6d87995-becb-45b7-9834-39fb2b78bafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:32:58.466832 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d87995-becb-45b7-9834-39fb2b78bafa None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:32:58.471957 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a632f9b2-f021-4d63-a552-6bbd2fd9e1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:32:58.473413 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632f9b2-f021-4d63-a552-6bbd2fd9e1ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:32:58.473703 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632f9b2-f021-4d63-a552-6bbd2fd9e1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:32:58.473971 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632f9b2-f021-4d63-a552-6bbd2fd9e1ed admin admin] PATH_INFO: `/v3/users/e664528e3e33426aae2d47640b57e079` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:32:58.477660 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a632f9b2-f021-4d63-a552-6bbd2fd9e1ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=e664528e3e33426aae2d47640b57e079)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:32:58.482069 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a632f9b2-f021-4d63-a552-6bbd2fd9e1ed admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:32:58.702222 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-a632f9b2-f021-4d63-a552-6bbd2fd9e1ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e664528e3e33426aae2d47640b57e079'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:32:58.702886 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-a632f9b2-f021-4d63-a552-6bbd2fd9e1ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e664528e3e33426aae2d47640b57e079'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:32:58.703523 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-a632f9b2-f021-4d63-a552-6bbd2fd9e1ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e664528e3e33426aae2d47640b57e079'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:32:58.711192 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-a632f9b2-f021-4d63-a552-6bbd2fd9e1ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e664528e3e33426aae2d47640b57e079'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:32:58.724046 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-a632f9b2-f021-4d63-a552-6bbd2fd9e1ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e664528e3e33426aae2d47640b57e079'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:32:58.838673 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1381/2792] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:32:58 2022] DELETE /identity/v3/users/e664528e3e33426aae2d47640b57e079 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:32:59.097931 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e1425-70eb-4788-9de2-9a43bd8056d1 service neutron] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:32:59.108855 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e1425-70eb-4788-9de2-9a43bd8056d1 service neutron] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:32:59.170141 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f100842e-0dfa-4993-b239-6acccf907a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:32:59.170782 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100842e-0dfa-4993-b239-6acccf907a4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:32:59.171071 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100842e-0dfa-4993-b239-6acccf907a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:32:59.171347 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100842e-0dfa-4993-b239-6acccf907a4c admin admin] PATH_INFO: `/v3/projects/e453baf77e6744eb8f001a1c1bb9d0d1` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:32:59.172559 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f100842e-0dfa-4993-b239-6acccf907a4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e453baf77e6744eb8f001a1c1bb9d0d1)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:32:59.174530 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f100842e-0dfa-4993-b239-6acccf907a4c admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:32:59.274502 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-f100842e-0dfa-4993-b239-6acccf907a4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e453baf77e6744eb8f001a1c1bb9d0d1', 'request_id': 'req-f100842e-0dfa-4993-b239-6acccf907a4c'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:32:59.281638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-f100842e-0dfa-4993-b239-6acccf907a4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e453baf77e6744eb8f001a1c1bb9d0d1', 'request_id': 'req-f100842e-0dfa-4993-b239-6acccf907a4c'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:32:59.346417 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-f100842e-0dfa-4993-b239-6acccf907a4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e453baf77e6744eb8f001a1c1bb9d0d1', 'request_id': 'req-f100842e-0dfa-4993-b239-6acccf907a4c'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:32:59.359260 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-f100842e-0dfa-4993-b239-6acccf907a4c admin admin] The token cache is being invalidate because project e453baf77e6744eb8f001a1c1bb9d0d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:32:59.359591 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-f100842e-0dfa-4993-b239-6acccf907a4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:32:59.360344 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-f100842e-0dfa-4993-b239-6acccf907a4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:32:59.361757 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1412/2793] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:32:59 2022] DELETE /identity/v3/projects/e453baf77e6744eb8f001a1c1bb9d0d1 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:32:59.367020 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a632f9b2-f021-4d63-a552-6bbd2fd9e1ed admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:32:59.368614 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-a632f9b2-f021-4d63-a552-6bbd2fd9e1ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:32:59.382162 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a632f9b2-f021-4d63-a552-6bbd2fd9e1ed admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:32:59.439688 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5613bf36-86cf-498d-b20b-83ad90a6727b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:32:59.440432 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613bf36-86cf-498d-b20b-83ad90a6727b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:32:59.440715 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613bf36-86cf-498d-b20b-83ad90a6727b admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:32:59.440996 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613bf36-86cf-498d-b20b-83ad90a6727b admin admin] PATH_INFO: `/v3/users/25515fa352914c2c939f69e23cae78b4` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:32:59.442313 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5613bf36-86cf-498d-b20b-83ad90a6727b admin admin] RBAC: Authorizing `identity:delete_user(user_id=25515fa352914c2c939f69e23cae78b4)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:32:59.444263 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5613bf36-86cf-498d-b20b-83ad90a6727b admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:32:59.618659 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-5613bf36-86cf-498d-b20b-83ad90a6727b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25515fa352914c2c939f69e23cae78b4'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:32:59.619334 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-5613bf36-86cf-498d-b20b-83ad90a6727b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25515fa352914c2c939f69e23cae78b4'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:32:59.619961 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-5613bf36-86cf-498d-b20b-83ad90a6727b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25515fa352914c2c939f69e23cae78b4'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:32:59.627284 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-5613bf36-86cf-498d-b20b-83ad90a6727b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25515fa352914c2c939f69e23cae78b4'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:32:59.639377 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-5613bf36-86cf-498d-b20b-83ad90a6727b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25515fa352914c2c939f69e23cae78b4'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:32:59.642337 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:32:59.723901 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1382/2794] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:32:59 2022] DELETE /identity/v3/users/25515fa352914c2c939f69e23cae78b4 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:32:59.951043 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f100842e-0dfa-4993-b239-6acccf907a4c admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:32:59.952648 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-f100842e-0dfa-4993-b239-6acccf907a4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:32:59.968611 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f100842e-0dfa-4993-b239-6acccf907a4c admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:00.027039 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1a718-d939-4d64-bbee-720b45b9c53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:00.027711 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1a718-d939-4d64-bbee-720b45b9c53a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:00.027999 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1a718-d939-4d64-bbee-720b45b9c53a admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:00.028279 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1a718-d939-4d64-bbee-720b45b9c53a admin admin] PATH_INFO: `/v3/projects/ac0531b9963648aeae3dc01ee6b37d61` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:00.029418 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1a718-d939-4d64-bbee-720b45b9c53a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac0531b9963648aeae3dc01ee6b37d61)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:00.031441 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1a718-d939-4d64-bbee-720b45b9c53a admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:00.053078 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:33:00.162422 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-f2a1a718-d939-4d64-bbee-720b45b9c53a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac0531b9963648aeae3dc01ee6b37d61', 'request_id': 'req-f2a1a718-d939-4d64-bbee-720b45b9c53a'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:00.168297 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-f2a1a718-d939-4d64-bbee-720b45b9c53a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac0531b9963648aeae3dc01ee6b37d61', 'request_id': 'req-f2a1a718-d939-4d64-bbee-720b45b9c53a'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:00.257813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-f2a1a718-d939-4d64-bbee-720b45b9c53a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac0531b9963648aeae3dc01ee6b37d61', 'request_id': 'req-f2a1a718-d939-4d64-bbee-720b45b9c53a'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:00.270158 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-f2a1a718-d939-4d64-bbee-720b45b9c53a admin admin] The token cache is being invalidate because project ac0531b9963648aeae3dc01ee6b37d61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:33:00.270445 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-f2a1a718-d939-4d64-bbee-720b45b9c53a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:00.271107 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-f2a1a718-d939-4d64-bbee-720b45b9c53a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:00.272619 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1413/2795] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:32:59 2022] DELETE /identity/v3/projects/ac0531b9963648aeae3dc01ee6b37d61 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:33:09.771675 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:33:10.266473 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:33:30.206731 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5613bf36-86cf-498d-b20b-83ad90a6727b admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:30.208182 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-5613bf36-86cf-498d-b20b-83ad90a6727b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:33:30.286274 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5613bf36-86cf-498d-b20b-83ad90a6727b admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:30.345738 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1a718-d939-4d64-bbee-720b45b9c53a admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:30.350984 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-f2a1a718-d939-4d64-bbee-720b45b9c53a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:33:30.355959 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7391a524-20c6-431e-a406-400083231aef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193637acadc14ec4a48ab79566ba3dab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:30.355959 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391a524-20c6-431e-a406-400083231aef service placement] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:30.355959 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391a524-20c6-431e-a406-400083231aef service placement] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:30.358854 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391a524-20c6-431e-a406-400083231aef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:30.360791 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-7391a524-20c6-431e-a406-400083231aef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:33:30.374851 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7391a524-20c6-431e-a406-400083231aef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:30.376780 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7391a524-20c6-431e-a406-400083231aef service placement] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:30.393215 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1383/2796] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:33:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:33:30.421267 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1a718-d939-4d64-bbee-720b45b9c53a admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:30.426372 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e76b3-962f-43c3-bf1d-91ca47707205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:30.427045 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e76b3-962f-43c3-bf1d-91ca47707205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:30.427333 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e76b3-962f-43c3-bf1d-91ca47707205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:30.427643 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e76b3-962f-43c3-bf1d-91ca47707205 admin admin] PATH_INFO: `/v3/users/27ed3f0ca7c44672a42aa774b02de88c` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:30.428813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e76b3-962f-43c3-bf1d-91ca47707205 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27ed3f0ca7c44672a42aa774b02de88c)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:30.431388 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e76b3-962f-43c3-bf1d-91ca47707205 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:30.611175 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-2b7e76b3-962f-43c3-bf1d-91ca47707205 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27ed3f0ca7c44672a42aa774b02de88c'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:30.618264 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-2b7e76b3-962f-43c3-bf1d-91ca47707205 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27ed3f0ca7c44672a42aa774b02de88c'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:30.618944 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-2b7e76b3-962f-43c3-bf1d-91ca47707205 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27ed3f0ca7c44672a42aa774b02de88c'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:30.631072 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-2b7e76b3-962f-43c3-bf1d-91ca47707205 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27ed3f0ca7c44672a42aa774b02de88c'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:30.699810 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-2b7e76b3-962f-43c3-bf1d-91ca47707205 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27ed3f0ca7c44672a42aa774b02de88c'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:30.706292 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1414/2797] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:33:30 2022] DELETE /identity/v3/users/27ed3f0ca7c44672a42aa774b02de88c => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:33:30.970551 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7391a524-20c6-431e-a406-400083231aef service placement] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:30.981532 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7391a524-20c6-431e-a406-400083231aef service placement] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:31.040603 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e44492-59bb-4a50-bc31-0fd0a657ae52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:31.041279 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e44492-59bb-4a50-bc31-0fd0a657ae52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:31.041562 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e44492-59bb-4a50-bc31-0fd0a657ae52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:31.041837 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e44492-59bb-4a50-bc31-0fd0a657ae52 admin admin] PATH_INFO: `/v3/projects/86b496e7df3e4e5d8781c49101c619c2` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:31.042963 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e44492-59bb-4a50-bc31-0fd0a657ae52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86b496e7df3e4e5d8781c49101c619c2)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:31.044893 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e44492-59bb-4a50-bc31-0fd0a657ae52 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:31.159619 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:33:31.169165 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-99e44492-59bb-4a50-bc31-0fd0a657ae52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86b496e7df3e4e5d8781c49101c619c2', 'request_id': 'req-99e44492-59bb-4a50-bc31-0fd0a657ae52'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:31.174421 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-99e44492-59bb-4a50-bc31-0fd0a657ae52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86b496e7df3e4e5d8781c49101c619c2', 'request_id': 'req-99e44492-59bb-4a50-bc31-0fd0a657ae52'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:31.243392 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-99e44492-59bb-4a50-bc31-0fd0a657ae52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86b496e7df3e4e5d8781c49101c619c2', 'request_id': 'req-99e44492-59bb-4a50-bc31-0fd0a657ae52'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:31.256217 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-99e44492-59bb-4a50-bc31-0fd0a657ae52 admin admin] The token cache is being invalidate because project 86b496e7df3e4e5d8781c49101c619c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:33:31.256749 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-99e44492-59bb-4a50-bc31-0fd0a657ae52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:31.257405 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-99e44492-59bb-4a50-bc31-0fd0a657ae52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:31.259726 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1384/2798] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:33:30 2022] DELETE /identity/v3/projects/86b496e7df3e4e5d8781c49101c619c2 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:33:31.264707 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e398c587-7e51-40a7-a15f-f054b06ba7f4 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:31.264997 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e398c587-7e51-40a7-a15f-f054b06ba7f4 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:31.265297 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e398c587-7e51-40a7-a15f-f054b06ba7f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:31.285358 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-e398c587-7e51-40a7-a15f-f054b06ba7f4 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:33:31.286596 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-e398c587-7e51-40a7-a15f-f054b06ba7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:33:31.350032 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1415/2799] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:33:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:33:31.357843 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f53f4-bfe0-4e31-b165-a68464ac9702 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:31.358132 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f53f4-bfe0-4e31-b165-a68464ac9702 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:31.358411 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f53f4-bfe0-4e31-b165-a68464ac9702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:31.378957 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-653f53f4-bfe0-4e31-b165-a68464ac9702 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:33:31.380266 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-653f53f4-bfe0-4e31-b165-a68464ac9702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:33:31.389623 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1385/2800] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:33:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:33:31.393436 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e398c587-7e51-40a7-a15f-f054b06ba7f4 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:31.403740 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e398c587-7e51-40a7-a15f-f054b06ba7f4 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:31.408541 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd395818-6ac7-41f8-8ee5-be17192c1b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:31.409258 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd395818-6ac7-41f8-8ee5-be17192c1b53 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:31.409537 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd395818-6ac7-41f8-8ee5-be17192c1b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:31.409812 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd395818-6ac7-41f8-8ee5-be17192c1b53 admin admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:31.410455 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd395818-6ac7-41f8-8ee5-be17192c1b53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:33:31.410765 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd395818-6ac7-41f8-8ee5-be17192c1b53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:31.412490 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd395818-6ac7-41f8-8ee5-be17192c1b53 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:31.424371 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1416/2801] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:33:31.429033 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653f53f4-bfe0-4e31-b165-a68464ac9702 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:31.440285 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653f53f4-bfe0-4e31-b165-a68464ac9702 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:31.445919 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eeab7d-4c44-4156-b07b-0ccbd6ccc967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:31.446561 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eeab7d-4c44-4156-b07b-0ccbd6ccc967 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:31.446841 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eeab7d-4c44-4156-b07b-0ccbd6ccc967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:31.447115 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eeab7d-4c44-4156-b07b-0ccbd6ccc967 admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:31.447793 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eeab7d-4c44-4156-b07b-0ccbd6ccc967 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:31.449746 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eeab7d-4c44-4156-b07b-0ccbd6ccc967 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:31.547732 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1386/2802] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:33:31 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:33:31.551686 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd395818-6ac7-41f8-8ee5-be17192c1b53 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:31.562215 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd395818-6ac7-41f8-8ee5-be17192c1b53 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:31.565790 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:33:31.618184 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5060a0b-1ae1-4515-904c-8c7a30d1a6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:31.618786 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5060a0b-1ae1-4515-904c-8c7a30d1a6ad admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:31.619066 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5060a0b-1ae1-4515-904c-8c7a30d1a6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:31.619334 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5060a0b-1ae1-4515-904c-8c7a30d1a6ad admin admin] PATH_INFO: `/v3/users` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:31.619989 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5060a0b-1ae1-4515-904c-8c7a30d1a6ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:31.621884 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5060a0b-1ae1-4515-904c-8c7a30d1a6ad admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:31.746052 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1417/2803] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:33:31 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:33:31.750231 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eeab7d-4c44-4156-b07b-0ccbd6ccc967 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:31.761701 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eeab7d-4c44-4156-b07b-0ccbd6ccc967 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:31.766675 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fae47d5-7d85-4d6f-805a-981fe5ffebc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:31.767287 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae47d5-7d85-4d6f-805a-981fe5ffebc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:31.767591 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae47d5-7d85-4d6f-805a-981fe5ffebc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:31.767904 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae47d5-7d85-4d6f-805a-981fe5ffebc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:31.768576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fae47d5-7d85-4d6f-805a-981fe5ffebc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:31.770153 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fae47d5-7d85-4d6f-805a-981fe5ffebc9 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:31.780873 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1387/2804] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:33:31.787737 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5060a0b-1ae1-4515-904c-8c7a30d1a6ad admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:31.798377 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5060a0b-1ae1-4515-904c-8c7a30d1a6ad admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:31.803047 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01325895-3a55-4533-910a-763da2b43a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:31.803672 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01325895-3a55-4533-910a-763da2b43a2f admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:31.803955 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01325895-3a55-4533-910a-763da2b43a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:31.804232 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01325895-3a55-4533-910a-763da2b43a2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:31.805413 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01325895-3a55-4533-910a-763da2b43a2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:31.807036 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01325895-3a55-4533-910a-763da2b43a2f admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:31.818571 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1418/2805] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:33:31.823490 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fae47d5-7d85-4d6f-805a-981fe5ffebc9 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:31.833510 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fae47d5-7d85-4d6f-805a-981fe5ffebc9 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:31.838934 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a00d413-17f8-4887-94ea-702aa756abcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:31.839613 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a00d413-17f8-4887-94ea-702aa756abcf admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:31.839902 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a00d413-17f8-4887-94ea-702aa756abcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:31.840188 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a00d413-17f8-4887-94ea-702aa756abcf admin admin] PATH_INFO: `/v3/projects/04767bc06d674ddea586d44324065975/users/78621860936e4b1bbaf8219cc47a06b9/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:31.855432 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a00d413-17f8-4887-94ea-702aa756abcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=04767bc06d674ddea586d44324065975, user_id=78621860936e4b1bbaf8219cc47a06b9, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:31.858080 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a00d413-17f8-4887-94ea-702aa756abcf admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:31.922309 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1388/2806] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:33:31 2022] PUT /identity/v3/projects/04767bc06d674ddea586d44324065975/users/78621860936e4b1bbaf8219cc47a06b9/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:33:31.933434 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ad216-4ba0-4609-b9b5-f980e2ca47ff None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:31.933728 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ad216-4ba0-4609-b9b5-f980e2ca47ff None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:31.938265 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ad216-4ba0-4609-b9b5-f980e2ca47ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:31.962593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-bc4ad216-4ba0-4609-b9b5-f980e2ca47ff None None] MFA Rules not processed for user `78621860936e4b1bbaf8219cc47a06b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:33:31.963879 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-bc4ad216-4ba0-4609-b9b5-f980e2ca47ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:33:32.027894 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1419/2807] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:33:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:33:32.041324 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a00d413-17f8-4887-94ea-702aa756abcf admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:32.043686 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-9a00d413-17f8-4887-94ea-702aa756abcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:33:32.120422 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a00d413-17f8-4887-94ea-702aa756abcf admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:32.125845 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b61035-cd8b-494a-b95e-bf15170d4d40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:32.126453 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b61035-cd8b-494a-b95e-bf15170d4d40 service nova] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:32.126737 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b61035-cd8b-494a-b95e-bf15170d4d40 service nova] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:32.127023 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b61035-cd8b-494a-b95e-bf15170d4d40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:32.134770 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b61035-cd8b-494a-b95e-bf15170d4d40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:32.136431 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b61035-cd8b-494a-b95e-bf15170d4d40 service nova] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:32.145241 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1389/2808] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:33:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:33:42.024937 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=63,keystone:INSERT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:33:42.141169 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:DELETE=5,keystone:SELECT=76,keystone:INSERT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:33:42.715974 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ad216-4ba0-4609-b9b5-f980e2ca47ff None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:42.717373 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-bc4ad216-4ba0-4609-b9b5-f980e2ca47ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:33:42.788616 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ad216-4ba0-4609-b9b5-f980e2ca47ff None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:42.793695 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30fc29-4c89-45e3-a505-ae746255fae8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193637acadc14ec4a48ab79566ba3dab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:42.797276 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30fc29-4c89-45e3-a505-ae746255fae8 service placement] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:42.797276 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30fc29-4c89-45e3-a505-ae746255fae8 service placement] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:42.797276 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30fc29-4c89-45e3-a505-ae746255fae8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:42.797276 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-ae30fc29-4c89-45e3-a505-ae746255fae8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:33:42.809878 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae30fc29-4c89-45e3-a505-ae746255fae8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:42.811557 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae30fc29-4c89-45e3-a505-ae746255fae8 service placement] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:42.822685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1420/2809] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:33:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:33:43.962784 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead53541-8bef-41d0-9227-7d80cce7c86f None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:43.963306 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead53541-8bef-41d0-9227-7d80cce7c86f None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:43.963428 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead53541-8bef-41d0-9227-7d80cce7c86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:43.983439 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-ead53541-8bef-41d0-9227-7d80cce7c86f None None] MFA Rules not processed for user `3907830851364a73b4d3715280a132bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:33:43.984657 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-ead53541-8bef-41d0-9227-7d80cce7c86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:33:44.038180 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1390/2810] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:33:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:33:44.356215 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30fc29-4c89-45e3-a505-ae746255fae8 service placement] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:44.357693 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-ae30fc29-4c89-45e3-a505-ae746255fae8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:33:44.414288 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30fc29-4c89-45e3-a505-ae746255fae8 service placement] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:44.417988 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e62e4a-9ee4-4287-a92a-cef512f2d146 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:44.418598 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e62e4a-9ee4-4287-a92a-cef512f2d146 service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:44.418881 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e62e4a-9ee4-4287-a92a-cef512f2d146 service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:44.419158 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e62e4a-9ee4-4287-a92a-cef512f2d146 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:44.421041 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-e8e62e4a-9ee4-4287-a92a-cef512f2d146 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:33:44.431769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e62e4a-9ee4-4287-a92a-cef512f2d146 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:44.433327 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e62e4a-9ee4-4287-a92a-cef512f2d146 service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:44.442030 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1421/2811] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:33:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:33:44.552578 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead53541-8bef-41d0-9227-7d80cce7c86f None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:44.610924 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead53541-8bef-41d0-9227-7d80cce7c86f None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:44.615570 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd48d16-8a94-490d-9640-ea71aebc58b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:44.616423 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd48d16-8a94-490d-9640-ea71aebc58b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:44.616560 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd48d16-8a94-490d-9640-ea71aebc58b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:44.616748 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd48d16-8a94-490d-9640-ea71aebc58b6 admin admin] PATH_INFO: `/v3/users/78621860936e4b1bbaf8219cc47a06b9` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:44.617913 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd48d16-8a94-490d-9640-ea71aebc58b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78621860936e4b1bbaf8219cc47a06b9)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:44.620417 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd48d16-8a94-490d-9640-ea71aebc58b6 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:44.817502 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-2fd48d16-8a94-490d-9640-ea71aebc58b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78621860936e4b1bbaf8219cc47a06b9'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:44.818164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-2fd48d16-8a94-490d-9640-ea71aebc58b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78621860936e4b1bbaf8219cc47a06b9'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:44.818774 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-2fd48d16-8a94-490d-9640-ea71aebc58b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78621860936e4b1bbaf8219cc47a06b9'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:44.826220 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-2fd48d16-8a94-490d-9640-ea71aebc58b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78621860936e4b1bbaf8219cc47a06b9'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:44.838502 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-2fd48d16-8a94-490d-9640-ea71aebc58b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78621860936e4b1bbaf8219cc47a06b9'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:44.972504 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1391/2812] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:33:44 2022] DELETE /identity/v3/users/78621860936e4b1bbaf8219cc47a06b9 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:33:44.977163 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909f828-7270-4035-8f28-60fe60399b65 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:44.977454 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909f828-7270-4035-8f28-60fe60399b65 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:44.977728 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909f828-7270-4035-8f28-60fe60399b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:44.997758 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-a909f828-7270-4035-8f28-60fe60399b65 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:33:44.998970 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-a909f828-7270-4035-8f28-60fe60399b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:33:45.057717 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1422/2813] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:33:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:33:45.067631 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd48d16-8a94-490d-9640-ea71aebc58b6 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:45.069111 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-2fd48d16-8a94-490d-9640-ea71aebc58b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:33:45.096995 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:33:45.126747 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd48d16-8a94-490d-9640-ea71aebc58b6 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:45.131185 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba697f-fd3d-4716-bb16-ee3634e1690d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:45.131806 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba697f-fd3d-4716-bb16-ee3634e1690d service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:45.132087 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba697f-fd3d-4716-bb16-ee3634e1690d service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:45.132362 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba697f-fd3d-4716-bb16-ee3634e1690d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:45.142581 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba697f-fd3d-4716-bb16-ee3634e1690d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:45.144110 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba697f-fd3d-4716-bb16-ee3634e1690d service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:45.153958 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1392/2814] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:33:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:33:45.573315 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a909f828-7270-4035-8f28-60fe60399b65 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:45.577891 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-a909f828-7270-4035-8f28-60fe60399b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:33:45.586902 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a909f828-7270-4035-8f28-60fe60399b65 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:45.593097 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9953270e-a07e-4a07-93af-eca2a2774bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:45.594084 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953270e-a07e-4a07-93af-eca2a2774bbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:45.594373 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953270e-a07e-4a07-93af-eca2a2774bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:45.594646 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953270e-a07e-4a07-93af-eca2a2774bbf admin admin] PATH_INFO: `/v3/projects/04767bc06d674ddea586d44324065975` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:45.595878 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9953270e-a07e-4a07-93af-eca2a2774bbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=04767bc06d674ddea586d44324065975)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:45.598610 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9953270e-a07e-4a07-93af-eca2a2774bbf admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:45.731111 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:33:45.745881 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-9953270e-a07e-4a07-93af-eca2a2774bbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04767bc06d674ddea586d44324065975', 'request_id': 'req-9953270e-a07e-4a07-93af-eca2a2774bbf'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:45.751329 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-9953270e-a07e-4a07-93af-eca2a2774bbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04767bc06d674ddea586d44324065975', 'request_id': 'req-9953270e-a07e-4a07-93af-eca2a2774bbf'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:45.857893 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-9953270e-a07e-4a07-93af-eca2a2774bbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04767bc06d674ddea586d44324065975', 'request_id': 'req-9953270e-a07e-4a07-93af-eca2a2774bbf'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:45.870209 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-9953270e-a07e-4a07-93af-eca2a2774bbf admin admin] The token cache is being invalidate because project 04767bc06d674ddea586d44324065975 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:33:45.870505 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-9953270e-a07e-4a07-93af-eca2a2774bbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:45.871609 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-9953270e-a07e-4a07-93af-eca2a2774bbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:45.878049 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1423/2815] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:33:45 2022] DELETE /identity/v3/projects/04767bc06d674ddea586d44324065975 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:33:45.879186 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239656f1-f43f-424e-ae3d-2af128f1649b None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:45.879586 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239656f1-f43f-424e-ae3d-2af128f1649b None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:45.879935 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239656f1-f43f-424e-ae3d-2af128f1649b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:45.900487 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-239656f1-f43f-424e-ae3d-2af128f1649b None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:33:45.901716 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-239656f1-f43f-424e-ae3d-2af128f1649b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:33:45.959834 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1393/2816] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:33:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:33:45.968614 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e5476-35b8-4d38-a9b0-ba0d0bb2609b None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:45.968910 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e5476-35b8-4d38-a9b0-ba0d0bb2609b None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:45.969193 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e5476-35b8-4d38-a9b0-ba0d0bb2609b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:45.988835 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-ea3e5476-35b8-4d38-a9b0-ba0d0bb2609b None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:33:45.990062 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-ea3e5476-35b8-4d38-a9b0-ba0d0bb2609b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:33:45.998049 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1424/2817] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:33:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:33:46.002447 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239656f1-f43f-424e-ae3d-2af128f1649b None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:46.012857 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239656f1-f43f-424e-ae3d-2af128f1649b None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:46.017468 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d418b56c-30fb-4a51-ae13-faad4c03ea2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:46.018070 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418b56c-30fb-4a51-ae13-faad4c03ea2b admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:46.018355 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418b56c-30fb-4a51-ae13-faad4c03ea2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:46.018657 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418b56c-30fb-4a51-ae13-faad4c03ea2b admin admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:46.019303 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d418b56c-30fb-4a51-ae13-faad4c03ea2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:33:46.019640 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d418b56c-30fb-4a51-ae13-faad4c03ea2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:46.021236 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d418b56c-30fb-4a51-ae13-faad4c03ea2b admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:46.033434 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1394/2818] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:33:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:33:46.037680 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e5476-35b8-4d38-a9b0-ba0d0bb2609b None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:46.047132 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e5476-35b8-4d38-a9b0-ba0d0bb2609b None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:46.051872 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e0fff-3b00-4bdc-8bda-d616aad2e995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:46.052470 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e0fff-3b00-4bdc-8bda-d616aad2e995 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:46.052755 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e0fff-3b00-4bdc-8bda-d616aad2e995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:46.053039 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e0fff-3b00-4bdc-8bda-d616aad2e995 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:46.053663 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63e0fff-3b00-4bdc-8bda-d616aad2e995 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:46.055580 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63e0fff-3b00-4bdc-8bda-d616aad2e995 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:46.126455 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1425/2819] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:33:46 2022] POST /identity/v3/projects => generated 381 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:33:46.128928 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d418b56c-30fb-4a51-ae13-faad4c03ea2b admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:46.140028 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d418b56c-30fb-4a51-ae13-faad4c03ea2b admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:46.196614 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1238fde-b193-4504-a613-fad314fe365a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:46.197875 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1238fde-b193-4504-a613-fad314fe365a admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:46.198160 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1238fde-b193-4504-a613-fad314fe365a admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:46.198720 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1238fde-b193-4504-a613-fad314fe365a admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:46.199361 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1238fde-b193-4504-a613-fad314fe365a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:46.201301 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1238fde-b193-4504-a613-fad314fe365a admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:46.310381 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1395/2820] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:33:46 2022] POST /identity/v3/users => generated 291 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:33:46.314615 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e0fff-3b00-4bdc-8bda-d616aad2e995 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:46.328540 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e0fff-3b00-4bdc-8bda-d616aad2e995 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:46.335071 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da008789-8c05-410c-bffc-d4356ac059b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:46.335738 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da008789-8c05-410c-bffc-d4356ac059b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:46.336029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da008789-8c05-410c-bffc-d4356ac059b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:46.336309 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da008789-8c05-410c-bffc-d4356ac059b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:46.336968 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da008789-8c05-410c-bffc-d4356ac059b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:46.338953 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da008789-8c05-410c-bffc-d4356ac059b2 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:46.351542 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1426/2821] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:33:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:33:46.355775 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1238fde-b193-4504-a613-fad314fe365a admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:46.366026 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1238fde-b193-4504-a613-fad314fe365a admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:46.371284 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25927bd2-e5d3-4b04-a044-3ad8b0b0db3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:46.372007 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25927bd2-e5d3-4b04-a044-3ad8b0b0db3e admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:46.372405 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25927bd2-e5d3-4b04-a044-3ad8b0b0db3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:46.372734 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25927bd2-e5d3-4b04-a044-3ad8b0b0db3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:46.373486 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25927bd2-e5d3-4b04-a044-3ad8b0b0db3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:46.375163 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25927bd2-e5d3-4b04-a044-3ad8b0b0db3e admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:46.386803 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1396/2822] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:33:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:33:46.391530 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da008789-8c05-410c-bffc-d4356ac059b2 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:46.402715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da008789-8c05-410c-bffc-d4356ac059b2 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:46.407578 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd76e9b-25b4-4e1a-badb-a917c8b1d387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:46.408247 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd76e9b-25b4-4e1a-badb-a917c8b1d387 admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:46.408532 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd76e9b-25b4-4e1a-badb-a917c8b1d387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:46.408810 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd76e9b-25b4-4e1a-badb-a917c8b1d387 admin admin] PATH_INFO: `/v3/projects/31c87d86cfed4f3f81bc37fe889307c3/users/0ea4a0fbd65f4afbbf70331151c09531/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:46.426346 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd76e9b-25b4-4e1a-badb-a917c8b1d387 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31c87d86cfed4f3f81bc37fe889307c3, user_id=0ea4a0fbd65f4afbbf70331151c09531, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:46.429044 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd76e9b-25b4-4e1a-badb-a917c8b1d387 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:46.501841 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1427/2823] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:33:46 2022] PUT /identity/v3/projects/31c87d86cfed4f3f81bc37fe889307c3/users/0ea4a0fbd65f4afbbf70331151c09531/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:33:46.513657 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff05bb-d738-4aef-90b6-d66df7099fa9 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:46.513947 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff05bb-d738-4aef-90b6-d66df7099fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:46.514230 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff05bb-d738-4aef-90b6-d66df7099fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:46.543172 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-a5ff05bb-d738-4aef-90b6-d66df7099fa9 None None] MFA Rules not processed for user `0ea4a0fbd65f4afbbf70331151c09531`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:33:46.544418 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-a5ff05bb-d738-4aef-90b6-d66df7099fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:33:46.564927 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:33:46.610788 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1397/2824] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:33:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:33:46.616516 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd76e9b-25b4-4e1a-badb-a917c8b1d387 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:46.680129 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd76e9b-25b4-4e1a-badb-a917c8b1d387 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:46.685331 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b185468-d922-406b-b275-25f9398221d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:46.686047 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b185468-d922-406b-b275-25f9398221d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:46.686346 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b185468-d922-406b-b275-25f9398221d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:46.686606 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b185468-d922-406b-b275-25f9398221d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:46.687241 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b185468-d922-406b-b275-25f9398221d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:46.689200 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b185468-d922-406b-b275-25f9398221d4 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:46.821495 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1428/2825] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:33:46 2022] POST /identity/v3/projects => generated 381 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:33:46.825626 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff05bb-d738-4aef-90b6-d66df7099fa9 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:46.836524 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff05bb-d738-4aef-90b6-d66df7099fa9 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:46.894735 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df3b18-832d-4d75-97fa-95b07e1dbcb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:46.895342 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df3b18-832d-4d75-97fa-95b07e1dbcb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:46.895659 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df3b18-832d-4d75-97fa-95b07e1dbcb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:46.895937 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df3b18-832d-4d75-97fa-95b07e1dbcb8 admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:46.896580 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df3b18-832d-4d75-97fa-95b07e1dbcb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:46.898493 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df3b18-832d-4d75-97fa-95b07e1dbcb8 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:47.006611 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1398/2826] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:33:46 2022] POST /identity/v3/users => generated 297 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:33:47.010594 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b185468-d922-406b-b275-25f9398221d4 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:47.021130 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b185468-d922-406b-b275-25f9398221d4 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:47.026406 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e49db6-4629-4cf0-900e-cb2c652bd5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:47.027003 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e49db6-4629-4cf0-900e-cb2c652bd5c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:47.027298 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e49db6-4629-4cf0-900e-cb2c652bd5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:47.027608 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e49db6-4629-4cf0-900e-cb2c652bd5c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:47.028256 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e49db6-4629-4cf0-900e-cb2c652bd5c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:47.029846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e49db6-4629-4cf0-900e-cb2c652bd5c4 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:47.042214 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1429/2827] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:33:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:33:47.046402 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df3b18-832d-4d75-97fa-95b07e1dbcb8 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:47.057159 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df3b18-832d-4d75-97fa-95b07e1dbcb8 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:47.061757 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f91b1d-c0b0-43c7-9096-838bbe2cfab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:47.062398 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f91b1d-c0b0-43c7-9096-838bbe2cfab9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:47.062684 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f91b1d-c0b0-43c7-9096-838bbe2cfab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:47.062963 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f91b1d-c0b0-43c7-9096-838bbe2cfab9 admin admin] PATH_INFO: `/v3/projects/63987d7a3aeb41a8b3f52a38165612e3/users/b962480df7c54998ba245646b3ddf484/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:47.078275 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f91b1d-c0b0-43c7-9096-838bbe2cfab9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63987d7a3aeb41a8b3f52a38165612e3, user_id=b962480df7c54998ba245646b3ddf484, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:47.081021 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f91b1d-c0b0-43c7-9096-838bbe2cfab9 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:47.143972 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1399/2828] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:33:47 2022] PUT /identity/v3/projects/63987d7a3aeb41a8b3f52a38165612e3/users/b962480df7c54998ba245646b3ddf484/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:33:47.148034 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e49db6-4629-4cf0-900e-cb2c652bd5c4 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:47.158216 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e49db6-4629-4cf0-900e-cb2c652bd5c4 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:47.184373 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:33:47.217303 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3883dfa7-4816-48ae-b5c1-71f81e3eeb2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:47.218334 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883dfa7-4816-48ae-b5c1-71f81e3eeb2a admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:47.218334 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883dfa7-4816-48ae-b5c1-71f81e3eeb2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:47.218575 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883dfa7-4816-48ae-b5c1-71f81e3eeb2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:47.219128 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3883dfa7-4816-48ae-b5c1-71f81e3eeb2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:47.222188 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3883dfa7-4816-48ae-b5c1-71f81e3eeb2a admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:47.232746 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1430/2829] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:33:47 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:33:47.236979 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f91b1d-c0b0-43c7-9096-838bbe2cfab9 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:47.248343 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f91b1d-c0b0-43c7-9096-838bbe2cfab9 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:47.253198 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147fc001-4c19-40c0-a3a7-52249d75cd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:47.253896 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147fc001-4c19-40c0-a3a7-52249d75cd8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:47.254180 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147fc001-4c19-40c0-a3a7-52249d75cd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:47.263127 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147fc001-4c19-40c0-a3a7-52249d75cd8a admin admin] PATH_INFO: `/v3/projects/63987d7a3aeb41a8b3f52a38165612e3/users/b962480df7c54998ba245646b3ddf484/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:47.263127 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147fc001-4c19-40c0-a3a7-52249d75cd8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=63987d7a3aeb41a8b3f52a38165612e3, user_id=b962480df7c54998ba245646b3ddf484, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:47.263127 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147fc001-4c19-40c0-a3a7-52249d75cd8a admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:47.269839 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1400/2830] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:33:47 2022] PUT /identity/v3/projects/63987d7a3aeb41a8b3f52a38165612e3/users/b962480df7c54998ba245646b3ddf484/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:33:47.333149 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebb2b0-d5e1-48d4-99ee-3d7c73298167 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:47.333441 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebb2b0-d5e1-48d4-99ee-3d7c73298167 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:47.333724 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebb2b0-d5e1-48d4-99ee-3d7c73298167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:47.362934 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-47ebb2b0-d5e1-48d4-99ee-3d7c73298167 None None] MFA Rules not processed for user `b962480df7c54998ba245646b3ddf484`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:33:47.364216 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-47ebb2b0-d5e1-48d4-99ee-3d7c73298167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:33:47.439093 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1431/2831] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:33:47 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:33:47.451360 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147fc001-4c19-40c0-a3a7-52249d75cd8a admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:47.453082 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-147fc001-4c19-40c0-a3a7-52249d75cd8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:33:47.533456 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147fc001-4c19-40c0-a3a7-52249d75cd8a admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:47.537173 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19d0dd-7ae5-46d3-a19a-84080fb66382 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:47.537776 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19d0dd-7ae5-46d3-a19a-84080fb66382 service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:47.538061 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19d0dd-7ae5-46d3-a19a-84080fb66382 service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:47.538344 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19d0dd-7ae5-46d3-a19a-84080fb66382 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:47.547904 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd19d0dd-7ae5-46d3-a19a-84080fb66382 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:47.549327 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd19d0dd-7ae5-46d3-a19a-84080fb66382 service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:47.553206 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:33:47.559066 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1401/2832] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:33:47 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:33:48.513046 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ebb2b0-d5e1-48d4-99ee-3d7c73298167 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:48.522748 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ebb2b0-d5e1-48d4-99ee-3d7c73298167 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:48.527757 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6f1f22-536f-4843-9b33-9b431c8c9609 tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b962480df7c54998ba245646b3ddf484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63987d7a3aeb41a8b3f52a38165612e3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:48.528365 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f1f22-536f-4843-9b33-9b431c8c9609 tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:48.528654 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f1f22-536f-4843-9b33-9b431c8c9609 tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:48.528929 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f1f22-536f-4843-9b33-9b431c8c9609 tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:48.529583 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6f1f22-536f-4843-9b33-9b431c8c9609 tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:33:48.530020 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6f1f22-536f-4843-9b33-9b431c8c9609 tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:48.531559 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6f1f22-536f-4843-9b33-9b431c8c9609 tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:48.543980 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1432/2833] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:33:48.549831 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19d0dd-7ae5-46d3-a19a-84080fb66382 service cinder] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:48.560139 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19d0dd-7ae5-46d3-a19a-84080fb66382 service cinder] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:48.565415 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6b7353-6a67-4789-80b5-37f6979ba8ba tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b962480df7c54998ba245646b3ddf484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63987d7a3aeb41a8b3f52a38165612e3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:48.566056 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b7353-6a67-4789-80b5-37f6979ba8ba tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:48.566344 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b7353-6a67-4789-80b5-37f6979ba8ba tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:48.570668 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b7353-6a67-4789-80b5-37f6979ba8ba tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:48.570668 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6b7353-6a67-4789-80b5-37f6979ba8ba tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:48.570668 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6b7353-6a67-4789-80b5-37f6979ba8ba tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:48.668039 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1402/2834] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:33:48 2022] POST /identity/v3/projects => generated 353 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:33:48.673799 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6f1f22-536f-4843-9b33-9b431c8c9609 tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:48.685060 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6f1f22-536f-4843-9b33-9b431c8c9609 tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:48.745272 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f64ee3-a630-462e-b1de-251ec1ee8b8f tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b962480df7c54998ba245646b3ddf484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63987d7a3aeb41a8b3f52a38165612e3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:48.745887 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f64ee3-a630-462e-b1de-251ec1ee8b8f tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:48.746175 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f64ee3-a630-462e-b1de-251ec1ee8b8f tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:48.746455 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f64ee3-a630-462e-b1de-251ec1ee8b8f tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] PATH_INFO: `/v3/domains` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:48.747099 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f64ee3-a630-462e-b1de-251ec1ee8b8f tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81863) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:33:48.747409 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f64ee3-a630-462e-b1de-251ec1ee8b8f tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:48.749035 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f64ee3-a630-462e-b1de-251ec1ee8b8f tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:48.761226 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1433/2835] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:33:48.801869 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6b7353-6a67-4789-80b5-37f6979ba8ba tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:48.813320 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6b7353-6a67-4789-80b5-37f6979ba8ba tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:48.817903 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d94ea-c89f-4884-88ef-a1a6e9abcf46 tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b962480df7c54998ba245646b3ddf484', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63987d7a3aeb41a8b3f52a38165612e3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:48.818555 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d94ea-c89f-4884-88ef-a1a6e9abcf46 tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:48.818840 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d94ea-c89f-4884-88ef-a1a6e9abcf46 tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:48.819114 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d94ea-c89f-4884-88ef-a1a6e9abcf46 tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] PATH_INFO: `/v3/projects/a124a7afce7a4694b0704898152c725a` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:48.820285 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940d94ea-c89f-4884-88ef-a1a6e9abcf46 tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a124a7afce7a4694b0704898152c725a)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:48.822288 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940d94ea-c89f-4884-88ef-a1a6e9abcf46 tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:48.975060 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-940d94ea-c89f-4884-88ef-a1a6e9abcf46 tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a124a7afce7a4694b0704898152c725a', 'request_id': 'req-940d94ea-c89f-4884-88ef-a1a6e9abcf46'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:48.981303 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-940d94ea-c89f-4884-88ef-a1a6e9abcf46 tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a124a7afce7a4694b0704898152c725a', 'request_id': 'req-940d94ea-c89f-4884-88ef-a1a6e9abcf46'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:49.053983 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-940d94ea-c89f-4884-88ef-a1a6e9abcf46 tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a124a7afce7a4694b0704898152c725a', 'request_id': 'req-940d94ea-c89f-4884-88ef-a1a6e9abcf46'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:49.066007 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-940d94ea-c89f-4884-88ef-a1a6e9abcf46 tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] The token cache is being invalidate because project a124a7afce7a4694b0704898152c725a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:33:49.066299 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-940d94ea-c89f-4884-88ef-a1a6e9abcf46 tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:49.067018 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-940d94ea-c89f-4884-88ef-a1a6e9abcf46 tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:49.068532 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1403/2836] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:33:48 2022] DELETE /identity/v3/projects/a124a7afce7a4694b0704898152c725a => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:33:49.560610 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f64ee3-a630-462e-b1de-251ec1ee8b8f tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:49.563026 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-c0f64ee3-a630-462e-b1de-251ec1ee8b8f tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:33:49.594822 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:33:49.636746 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f64ee3-a630-462e-b1de-251ec1ee8b8f tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:49.702687 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d94ea-c89f-4884-88ef-a1a6e9abcf46 tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:49.705924 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-940d94ea-c89f-4884-88ef-a1a6e9abcf46 tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:33:49.738972 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa6311c-d8ec-4431-96dd-4fd2f52dd7c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:49.739947 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa6311c-d8ec-4431-96dd-4fd2f52dd7c9 service nova] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:49.740639 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa6311c-d8ec-4431-96dd-4fd2f52dd7c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:49.741289 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa6311c-d8ec-4431-96dd-4fd2f52dd7c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:49.743939 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-4fa6311c-d8ec-4431-96dd-4fd2f52dd7c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:33:49.802008 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d94ea-c89f-4884-88ef-a1a6e9abcf46 tempest-VolumeQuotaClassesTest-1174258618 tempest-VolumeQuotaClassesTest-1174258618-project-admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:49.808514 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab14aa66-abfc-4f71-a616-3b6bc69789e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:49.809172 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14aa66-abfc-4f71-a616-3b6bc69789e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:49.809457 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14aa66-abfc-4f71-a616-3b6bc69789e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:49.809819 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14aa66-abfc-4f71-a616-3b6bc69789e7 admin admin] PATH_INFO: `/v3/users/0ea4a0fbd65f4afbbf70331151c09531` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:49.811026 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab14aa66-abfc-4f71-a616-3b6bc69789e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ea4a0fbd65f4afbbf70331151c09531)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:49.813073 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab14aa66-abfc-4f71-a616-3b6bc69789e7 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:49.839947 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa6311c-d8ec-4431-96dd-4fd2f52dd7c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:49.841661 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa6311c-d8ec-4431-96dd-4fd2f52dd7c9 service nova] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:49.852433 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1434/2837] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:33:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:33:50.033565 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-ab14aa66-abfc-4f71-a616-3b6bc69789e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ea4a0fbd65f4afbbf70331151c09531'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:50.034341 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-ab14aa66-abfc-4f71-a616-3b6bc69789e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ea4a0fbd65f4afbbf70331151c09531'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:50.034959 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-ab14aa66-abfc-4f71-a616-3b6bc69789e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ea4a0fbd65f4afbbf70331151c09531'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:50.042354 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-ab14aa66-abfc-4f71-a616-3b6bc69789e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ea4a0fbd65f4afbbf70331151c09531'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:50.046801 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=84,keystone:INSERT=2,keystone:DELETE=14 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:33:50.056658 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-ab14aa66-abfc-4f71-a616-3b6bc69789e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ea4a0fbd65f4afbbf70331151c09531'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:50.166921 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1404/2838] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:33:49 2022] DELETE /identity/v3/users/0ea4a0fbd65f4afbbf70331151c09531 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:33:50.171673 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca8d9b-d825-4949-ba6f-1bcffc0cc65c None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:50.171969 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca8d9b-d825-4949-ba6f-1bcffc0cc65c None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:50.172255 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca8d9b-d825-4949-ba6f-1bcffc0cc65c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:50.193326 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-67ca8d9b-d825-4949-ba6f-1bcffc0cc65c None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:33:50.194559 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-67ca8d9b-d825-4949-ba6f-1bcffc0cc65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:33:50.255394 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1435/2839] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:33:50.265192 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab14aa66-abfc-4f71-a616-3b6bc69789e7 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:50.266625 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-ab14aa66-abfc-4f71-a616-3b6bc69789e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:33:50.324409 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab14aa66-abfc-4f71-a616-3b6bc69789e7 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:50.328588 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f3feab-5c8c-42d5-9c82-9a41b605b6e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:50.329193 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3feab-5c8c-42d5-9c82-9a41b605b6e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:50.329482 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3feab-5c8c-42d5-9c82-9a41b605b6e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:50.329761 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3feab-5c8c-42d5-9c82-9a41b605b6e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:50.339217 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f3feab-5c8c-42d5-9c82-9a41b605b6e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:50.340687 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f3feab-5c8c-42d5-9c82-9a41b605b6e7 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:50.350155 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1405/2840] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:33:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:33:50.782552 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca8d9b-d825-4949-ba6f-1bcffc0cc65c None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:50.787236 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-67ca8d9b-d825-4949-ba6f-1bcffc0cc65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:33:50.793505 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=100 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:33:50.798667 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca8d9b-d825-4949-ba6f-1bcffc0cc65c None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:50.803666 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94008aaa-3388-4a88-b812-af2694c396fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:50.804292 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94008aaa-3388-4a88-b812-af2694c396fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:50.804579 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94008aaa-3388-4a88-b812-af2694c396fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:50.804855 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94008aaa-3388-4a88-b812-af2694c396fd admin admin] PATH_INFO: `/v3/projects/31c87d86cfed4f3f81bc37fe889307c3` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:50.806874 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94008aaa-3388-4a88-b812-af2694c396fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=31c87d86cfed4f3f81bc37fe889307c3)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:50.812992 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94008aaa-3388-4a88-b812-af2694c396fd admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:50.957813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-94008aaa-3388-4a88-b812-af2694c396fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31c87d86cfed4f3f81bc37fe889307c3', 'request_id': 'req-94008aaa-3388-4a88-b812-af2694c396fd'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:50.960099 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-94008aaa-3388-4a88-b812-af2694c396fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31c87d86cfed4f3f81bc37fe889307c3', 'request_id': 'req-94008aaa-3388-4a88-b812-af2694c396fd'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:51.036871 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-94008aaa-3388-4a88-b812-af2694c396fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31c87d86cfed4f3f81bc37fe889307c3', 'request_id': 'req-94008aaa-3388-4a88-b812-af2694c396fd'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:51.048452 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-94008aaa-3388-4a88-b812-af2694c396fd admin admin] The token cache is being invalidate because project 31c87d86cfed4f3f81bc37fe889307c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:33:51.054678 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-94008aaa-3388-4a88-b812-af2694c396fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:51.054678 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-94008aaa-3388-4a88-b812-af2694c396fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:51.054678 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1436/2841] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:33:50 2022] DELETE /identity/v3/projects/31c87d86cfed4f3f81bc37fe889307c3 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:33:51.057534 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f3feab-5c8c-42d5-9c82-9a41b605b6e7 service neutron] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:51.067197 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f3feab-5c8c-42d5-9c82-9a41b605b6e7 service neutron] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:51.122355 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977468e-e9c7-49a9-9d55-28831069c6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:51.122983 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977468e-e9c7-49a9-9d55-28831069c6f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:51.123269 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977468e-e9c7-49a9-9d55-28831069c6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:51.123578 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977468e-e9c7-49a9-9d55-28831069c6f3 admin admin] PATH_INFO: `/v3/users/b962480df7c54998ba245646b3ddf484` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:51.124705 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0977468e-e9c7-49a9-9d55-28831069c6f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b962480df7c54998ba245646b3ddf484)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:51.126607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0977468e-e9c7-49a9-9d55-28831069c6f3 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:51.273980 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-0977468e-e9c7-49a9-9d55-28831069c6f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b962480df7c54998ba245646b3ddf484'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:51.274615 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-0977468e-e9c7-49a9-9d55-28831069c6f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b962480df7c54998ba245646b3ddf484'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:51.275849 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-0977468e-e9c7-49a9-9d55-28831069c6f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b962480df7c54998ba245646b3ddf484'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:51.282956 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-0977468e-e9c7-49a9-9d55-28831069c6f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b962480df7c54998ba245646b3ddf484'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:51.295759 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-0977468e-e9c7-49a9-9d55-28831069c6f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b962480df7c54998ba245646b3ddf484'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:51.379933 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:33:51.385218 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1406/2842] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:33:51 2022] DELETE /identity/v3/users/b962480df7c54998ba245646b3ddf484 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:33:51.800768 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94008aaa-3388-4a88-b812-af2694c396fd admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:51.802820 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-94008aaa-3388-4a88-b812-af2694c396fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:33:51.820721 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94008aaa-3388-4a88-b812-af2694c396fd admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:51.884000 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efd8e6e-41d2-4115-af5e-b0e8f0e7440c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:51.885160 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd8e6e-41d2-4115-af5e-b0e8f0e7440c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:51.885446 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd8e6e-41d2-4115-af5e-b0e8f0e7440c admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:51.885717 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd8e6e-41d2-4115-af5e-b0e8f0e7440c admin admin] PATH_INFO: `/v3/projects/63987d7a3aeb41a8b3f52a38165612e3` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:51.886844 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efd8e6e-41d2-4115-af5e-b0e8f0e7440c admin admin] RBAC: Authorizing `identity:delete_project(project_id=63987d7a3aeb41a8b3f52a38165612e3)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:51.888838 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efd8e6e-41d2-4115-af5e-b0e8f0e7440c admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:52.019651 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-7efd8e6e-41d2-4115-af5e-b0e8f0e7440c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63987d7a3aeb41a8b3f52a38165612e3', 'request_id': 'req-7efd8e6e-41d2-4115-af5e-b0e8f0e7440c'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:52.025601 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-7efd8e6e-41d2-4115-af5e-b0e8f0e7440c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63987d7a3aeb41a8b3f52a38165612e3', 'request_id': 'req-7efd8e6e-41d2-4115-af5e-b0e8f0e7440c'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:52.105864 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-7efd8e6e-41d2-4115-af5e-b0e8f0e7440c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63987d7a3aeb41a8b3f52a38165612e3', 'request_id': 'req-7efd8e6e-41d2-4115-af5e-b0e8f0e7440c'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:52.119922 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-7efd8e6e-41d2-4115-af5e-b0e8f0e7440c admin admin] The token cache is being invalidate because project 63987d7a3aeb41a8b3f52a38165612e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:33:52.120681 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-7efd8e6e-41d2-4115-af5e-b0e8f0e7440c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:52.121430 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-7efd8e6e-41d2-4115-af5e-b0e8f0e7440c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:33:52.126699 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1437/2843] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:33:51 2022] DELETE /identity/v3/projects/63987d7a3aeb41a8b3f52a38165612e3 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:33:52.128348 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd205a-758e-4958-8163-465f31bd4560 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:52.129134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd205a-758e-4958-8163-465f31bd4560 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:52.129134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd205a-758e-4958-8163-465f31bd4560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:52.155811 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-33dd205a-758e-4958-8163-465f31bd4560 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:33:52.157192 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-33dd205a-758e-4958-8163-465f31bd4560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:33:52.230532 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1407/2844] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:33:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:33:52.238210 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7941a43c-0839-41ba-a13f-2f077885c374 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:52.238494 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7941a43c-0839-41ba-a13f-2f077885c374 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:52.238775 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7941a43c-0839-41ba-a13f-2f077885c374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:52.260699 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-7941a43c-0839-41ba-a13f-2f077885c374 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:33:52.261947 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-7941a43c-0839-41ba-a13f-2f077885c374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:33:52.270322 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1438/2845] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:33:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:33:52.275389 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd205a-758e-4958-8163-465f31bd4560 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:52.285396 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd205a-758e-4958-8163-465f31bd4560 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:52.291124 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc3808-e52d-4713-b1ea-18bfbe32395c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:52.291773 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc3808-e52d-4713-b1ea-18bfbe32395c admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:52.296501 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc3808-e52d-4713-b1ea-18bfbe32395c admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:52.296501 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc3808-e52d-4713-b1ea-18bfbe32395c admin admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:52.296501 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cc3808-e52d-4713-b1ea-18bfbe32395c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:33:52.296501 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cc3808-e52d-4713-b1ea-18bfbe32395c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:52.296501 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cc3808-e52d-4713-b1ea-18bfbe32395c admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:52.311458 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1408/2846] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:33:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 18 12:33:52.314102 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7941a43c-0839-41ba-a13f-2f077885c374 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:52.324788 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7941a43c-0839-41ba-a13f-2f077885c374 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:52.331707 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f870e1-1aff-415c-813f-ec927b5382cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:52.332336 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f870e1-1aff-415c-813f-ec927b5382cd admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:52.332833 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f870e1-1aff-415c-813f-ec927b5382cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:52.333124 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f870e1-1aff-415c-813f-ec927b5382cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:52.333761 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f870e1-1aff-415c-813f-ec927b5382cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:52.336007 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f870e1-1aff-415c-813f-ec927b5382cd admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:52.347132 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:33:52.450814 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1439/2847] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:33:52 2022] POST /identity/v3/projects => generated 371 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:33:52.454596 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc3808-e52d-4713-b1ea-18bfbe32395c admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:52.465507 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc3808-e52d-4713-b1ea-18bfbe32395c admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:52.529749 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb6563-8ccd-4eb5-af92-72fba055c9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:52.530438 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb6563-8ccd-4eb5-af92-72fba055c9e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:52.530794 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb6563-8ccd-4eb5-af92-72fba055c9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:52.531202 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb6563-8ccd-4eb5-af92-72fba055c9e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:52.531929 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bb6563-8ccd-4eb5-af92-72fba055c9e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:52.533916 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bb6563-8ccd-4eb5-af92-72fba055c9e2 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:52.624626 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1409/2848] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:33:52 2022] POST /identity/v3/users => generated 286 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:33:52.628820 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f870e1-1aff-415c-813f-ec927b5382cd admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:52.639927 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f870e1-1aff-415c-813f-ec927b5382cd admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:52.644945 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66dcd1e-6ceb-4da6-a889-621febf39b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:52.649748 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66dcd1e-6ceb-4da6-a889-621febf39b73 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:52.649748 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66dcd1e-6ceb-4da6-a889-621febf39b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:52.649748 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66dcd1e-6ceb-4da6-a889-621febf39b73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:52.649748 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66dcd1e-6ceb-4da6-a889-621febf39b73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:52.649748 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66dcd1e-6ceb-4da6-a889-621febf39b73 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:52.660188 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1440/2849] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:33:52.664461 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb6563-8ccd-4eb5-af92-72fba055c9e2 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:52.674934 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb6563-8ccd-4eb5-af92-72fba055c9e2 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:52.679752 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e6fc56-224d-4472-b178-ae9a818582b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:52.680359 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6fc56-224d-4472-b178-ae9a818582b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:52.680638 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6fc56-224d-4472-b178-ae9a818582b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:52.680909 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6fc56-224d-4472-b178-ae9a818582b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:52.681632 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e6fc56-224d-4472-b178-ae9a818582b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:52.683288 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e6fc56-224d-4472-b178-ae9a818582b4 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:52.694927 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1410/2850] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:33:52.699528 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66dcd1e-6ceb-4da6-a889-621febf39b73 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:52.709994 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66dcd1e-6ceb-4da6-a889-621febf39b73 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:52.715732 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6045dc6c-5649-42e0-bf1b-d599fa61ab86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:52.717359 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045dc6c-5649-42e0-bf1b-d599fa61ab86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:52.717674 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045dc6c-5649-42e0-bf1b-d599fa61ab86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:52.717929 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045dc6c-5649-42e0-bf1b-d599fa61ab86 admin admin] PATH_INFO: `/v3/projects/3e8fe8f5ca4d410b8fcdad9e420bd49a/users/863b316a47924d2290f5b054916afdea/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:52.733218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6045dc6c-5649-42e0-bf1b-d599fa61ab86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e8fe8f5ca4d410b8fcdad9e420bd49a, user_id=863b316a47924d2290f5b054916afdea, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:52.735963 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6045dc6c-5649-42e0-bf1b-d599fa61ab86 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:52.818740 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1441/2851] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:33:52 2022] PUT /identity/v3/projects/3e8fe8f5ca4d410b8fcdad9e420bd49a/users/863b316a47924d2290f5b054916afdea/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:33:52.824401 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9c9e3-7b65-4c66-9b8f-6df138275a15 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:52.824694 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9c9e3-7b65-4c66-9b8f-6df138275a15 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:52.824989 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9c9e3-7b65-4c66-9b8f-6df138275a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:52.844886 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-2ce9c9e3-7b65-4c66-9b8f-6df138275a15 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:33:52.846794 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-2ce9c9e3-7b65-4c66-9b8f-6df138275a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:33:52.910162 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1411/2852] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:33:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:33:52.920331 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6045dc6c-5649-42e0-bf1b-d599fa61ab86 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:52.922255 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-6045dc6c-5649-42e0-bf1b-d599fa61ab86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:33:52.983674 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6045dc6c-5649-42e0-bf1b-d599fa61ab86 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:52.987582 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed559398-76ab-4345-a7e6-25fe505bc8c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:52.988209 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed559398-76ab-4345-a7e6-25fe505bc8c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:52.988645 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed559398-76ab-4345-a7e6-25fe505bc8c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:52.988928 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed559398-76ab-4345-a7e6-25fe505bc8c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:52.999389 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed559398-76ab-4345-a7e6-25fe505bc8c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:53.001113 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed559398-76ab-4345-a7e6-25fe505bc8c8 service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:53.012450 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1442/2853] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:33:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:33:54.348987 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de34c8d-ce23-409d-8170-5151b2883d42 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:54.349942 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de34c8d-ce23-409d-8170-5151b2883d42 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:54.350592 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de34c8d-ce23-409d-8170-5151b2883d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:54.365848 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:33:54.382808 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-1de34c8d-ce23-409d-8170-5151b2883d42 None None] MFA Rules not processed for user `863b316a47924d2290f5b054916afdea`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:33:54.384613 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-1de34c8d-ce23-409d-8170-5151b2883d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:33:54.472341 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1412/2854] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:33:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:33:54.476462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed559398-76ab-4345-a7e6-25fe505bc8c8 service neutron] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:54.487574 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed559398-76ab-4345-a7e6-25fe505bc8c8 service neutron] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:54.492552 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ea47e-4779-46c3-97bf-37bb10e087c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:54.493175 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ea47e-4779-46c3-97bf-37bb10e087c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:54.493512 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ea47e-4779-46c3-97bf-37bb10e087c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:54.493767 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ea47e-4779-46c3-97bf-37bb10e087c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:54.494462 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641ea47e-4779-46c3-97bf-37bb10e087c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:54.496546 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641ea47e-4779-46c3-97bf-37bb10e087c1 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:54.588218 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1443/2855] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:33:54 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:33:54.589816 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de34c8d-ce23-409d-8170-5151b2883d42 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:54.603910 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de34c8d-ce23-409d-8170-5151b2883d42 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:54.663670 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf06297-0217-49b8-a1ec-082e01f686be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:54.664318 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf06297-0217-49b8-a1ec-082e01f686be admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:54.664602 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf06297-0217-49b8-a1ec-082e01f686be admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:54.664875 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf06297-0217-49b8-a1ec-082e01f686be admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:54.665551 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf06297-0217-49b8-a1ec-082e01f686be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:54.667482 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf06297-0217-49b8-a1ec-082e01f686be admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:54.757619 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1413/2856] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:33:54 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 18 12:33:54.762293 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ea47e-4779-46c3-97bf-37bb10e087c1 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:54.774538 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ea47e-4779-46c3-97bf-37bb10e087c1 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:54.780187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539afee-57b5-4119-97af-762daaca1a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:54.783596 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539afee-57b5-4119-97af-762daaca1a14 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:54.783596 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539afee-57b5-4119-97af-762daaca1a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:54.783596 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539afee-57b5-4119-97af-762daaca1a14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:54.783596 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a539afee-57b5-4119-97af-762daaca1a14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:54.784002 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a539afee-57b5-4119-97af-762daaca1a14 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:54.794866 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1444/2857] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:33:54.799244 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf06297-0217-49b8-a1ec-082e01f686be admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:54.809453 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf06297-0217-49b8-a1ec-082e01f686be admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:54.814336 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32737929-b3dd-43ef-b900-e9a36f583d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:54.815116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32737929-b3dd-43ef-b900-e9a36f583d5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:54.815401 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32737929-b3dd-43ef-b900-e9a36f583d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:54.815712 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32737929-b3dd-43ef-b900-e9a36f583d5c admin admin] PATH_INFO: `/v3/projects/10f6cbdea63f43c78e604a02940c6e7e/users/ab1fc36506f44cb9bd55857cec1a8c16/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:54.831122 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32737929-b3dd-43ef-b900-e9a36f583d5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=10f6cbdea63f43c78e604a02940c6e7e, user_id=ab1fc36506f44cb9bd55857cec1a8c16, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:54.833889 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32737929-b3dd-43ef-b900-e9a36f583d5c admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:54.900142 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1414/2858] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:33:54 2022] PUT /identity/v3/projects/10f6cbdea63f43c78e604a02940c6e7e/users/ab1fc36506f44cb9bd55857cec1a8c16/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:33:54.904323 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539afee-57b5-4119-97af-762daaca1a14 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:54.914271 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539afee-57b5-4119-97af-762daaca1a14 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:54.970363 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c52c7-ce4b-4bbb-817e-b23d929fd919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:54.970970 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c52c7-ce4b-4bbb-817e-b23d929fd919 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:54.971276 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c52c7-ce4b-4bbb-817e-b23d929fd919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:54.971592 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c52c7-ce4b-4bbb-817e-b23d929fd919 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:54.972231 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7c52c7-ce4b-4bbb-817e-b23d929fd919 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:54.973832 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7c52c7-ce4b-4bbb-817e-b23d929fd919 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:54.984000 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1445/2859] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:33:54.988231 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32737929-b3dd-43ef-b900-e9a36f583d5c admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:54.999138 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32737929-b3dd-43ef-b900-e9a36f583d5c admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:55.003788 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d9cc2c-cd6c-4b0e-8169-de0b55a739c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:55.004438 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9cc2c-cd6c-4b0e-8169-de0b55a739c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:55.004721 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9cc2c-cd6c-4b0e-8169-de0b55a739c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:55.004997 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9cc2c-cd6c-4b0e-8169-de0b55a739c3 admin admin] PATH_INFO: `/v3/projects/10f6cbdea63f43c78e604a02940c6e7e/users/ab1fc36506f44cb9bd55857cec1a8c16/roles/c53a0e13b58642f291c856efe1f0b87d` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:55.006946 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d9cc2c-cd6c-4b0e-8169-de0b55a739c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10f6cbdea63f43c78e604a02940c6e7e, user_id=ab1fc36506f44cb9bd55857cec1a8c16, role_id=c53a0e13b58642f291c856efe1f0b87d)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:55.009621 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d9cc2c-cd6c-4b0e-8169-de0b55a739c3 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:55.019891 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1415/2860] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:33:54 2022] PUT /identity/v3/projects/10f6cbdea63f43c78e604a02940c6e7e/users/ab1fc36506f44cb9bd55857cec1a8c16/roles/c53a0e13b58642f291c856efe1f0b87d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:33:56.242126 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80429f9-4fd4-42d6-89e1-ebcfaf93c70b None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:56.242643 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80429f9-4fd4-42d6-89e1-ebcfaf93c70b None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:56.242771 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80429f9-4fd4-42d6-89e1-ebcfaf93c70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:56.272823 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-a80429f9-4fd4-42d6-89e1-ebcfaf93c70b None None] MFA Rules not processed for user `ab1fc36506f44cb9bd55857cec1a8c16`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:33:56.274079 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-a80429f9-4fd4-42d6-89e1-ebcfaf93c70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:33:56.283334 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:33:56.344591 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1446/2861] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:33:56 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:33:56.355901 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d9cc2c-cd6c-4b0e-8169-de0b55a739c3 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:56.357585 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-d3d9cc2c-cd6c-4b0e-8169-de0b55a739c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:33:56.412472 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:33:56.418385 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d9cc2c-cd6c-4b0e-8169-de0b55a739c3 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:56.422783 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9bc71-5f22-43ce-af1a-edb2e13f8603 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:56.423430 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9bc71-5f22-43ce-af1a-edb2e13f8603 service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:56.423748 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9bc71-5f22-43ce-af1a-edb2e13f8603 service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:56.424022 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9bc71-5f22-43ce-af1a-edb2e13f8603 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:56.481649 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f9bc71-5f22-43ce-af1a-edb2e13f8603 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:56.483091 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f9bc71-5f22-43ce-af1a-edb2e13f8603 service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:56.492105 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1416/2862] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:33:56 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:33:58.238829 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80429f9-4fd4-42d6-89e1-ebcfaf93c70b None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:33:58.240298 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-a80429f9-4fd4-42d6-89e1-ebcfaf93c70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:33:58.252829 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80429f9-4fd4-42d6-89e1-ebcfaf93c70b None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:33:58.256435 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b487a-8937-4ca0-80fb-4d74bf41716d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:33:58.257043 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b487a-8937-4ca0-80fb-4d74bf41716d service cinder] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:33:58.257328 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b487a-8937-4ca0-80fb-4d74bf41716d service cinder] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:33:58.257606 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b487a-8937-4ca0-80fb-4d74bf41716d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:33:58.267206 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840b487a-8937-4ca0-80fb-4d74bf41716d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:33:58.268666 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840b487a-8937-4ca0-80fb-4d74bf41716d service cinder] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:33:58.278192 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1447/2863] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:33:58 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:34:06.498227 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=26 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:34:08.273989 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=31 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:34:11.207759 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9bc71-5f22-43ce-af1a-edb2e13f8603 service cinder] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:34:11.214049 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-63f9bc71-5f22-43ce-af1a-edb2e13f8603 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:34:11.305616 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9bc71-5f22-43ce-af1a-edb2e13f8603 service cinder] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:34:11.313850 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca984ae-fb78-4e95-bdac-685662dd46e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171c72a6b29e418bbb4ea5c6faffe3b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:34:11.340044 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca984ae-fb78-4e95-bdac-685662dd46e4 service nova] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:34:11.340044 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca984ae-fb78-4e95-bdac-685662dd46e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:34:11.340044 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca984ae-fb78-4e95-bdac-685662dd46e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:34:11.340044 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-bca984ae-fb78-4e95-bdac-685662dd46e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:34:11.340044 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca984ae-fb78-4e95-bdac-685662dd46e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:34:11.340044 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca984ae-fb78-4e95-bdac-685662dd46e4 service nova] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:34:11.345853 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1417/2864] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:34:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:34:13.624507 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b487a-8937-4ca0-80fb-4d74bf41716d service cinder] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:34:13.625947 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-840b487a-8937-4ca0-80fb-4d74bf41716d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:34:13.699287 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b487a-8937-4ca0-80fb-4d74bf41716d service cinder] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:34:13.704890 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bced74-fe72-415a-a204-f57c62b4d135 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193637acadc14ec4a48ab79566ba3dab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:34:13.705526 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bced74-fe72-415a-a204-f57c62b4d135 service placement] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:34:13.705813 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bced74-fe72-415a-a204-f57c62b4d135 service placement] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:34:13.706089 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bced74-fe72-415a-a204-f57c62b4d135 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:34:13.707865 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-57bced74-fe72-415a-a204-f57c62b4d135 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:34:13.720877 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bced74-fe72-415a-a204-f57c62b4d135 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:34:13.722518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bced74-fe72-415a-a204-f57c62b4d135 service placement] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:34:13.735304 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1448/2865] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:34:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:34:14.370083 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca984ae-fb78-4e95-bdac-685662dd46e4 service nova] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:34:14.435072 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca984ae-fb78-4e95-bdac-685662dd46e4 service nova] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:34:14.439543 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf6665-6cbf-4352-b0ae-dd92e5b45f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:34:14.440164 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf6665-6cbf-4352-b0ae-dd92e5b45f82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:34:14.440441 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf6665-6cbf-4352-b0ae-dd92e5b45f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:34:14.440713 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf6665-6cbf-4352-b0ae-dd92e5b45f82 admin admin] PATH_INFO: `/v3/users/863b316a47924d2290f5b054916afdea` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:34:14.441839 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf6665-6cbf-4352-b0ae-dd92e5b45f82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=863b316a47924d2290f5b054916afdea)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:34:14.443769 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf6665-6cbf-4352-b0ae-dd92e5b45f82 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:34:14.598252 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-f9bf6665-6cbf-4352-b0ae-dd92e5b45f82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '863b316a47924d2290f5b054916afdea'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:14.598916 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-f9bf6665-6cbf-4352-b0ae-dd92e5b45f82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '863b316a47924d2290f5b054916afdea'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:14.599555 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-f9bf6665-6cbf-4352-b0ae-dd92e5b45f82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '863b316a47924d2290f5b054916afdea'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:14.606526 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-f9bf6665-6cbf-4352-b0ae-dd92e5b45f82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '863b316a47924d2290f5b054916afdea'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:14.618625 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-f9bf6665-6cbf-4352-b0ae-dd92e5b45f82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '863b316a47924d2290f5b054916afdea'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:14.699888 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1418/2866] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:34:14 2022] DELETE /identity/v3/users/863b316a47924d2290f5b054916afdea => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:34:14.953035 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bced74-fe72-415a-a204-f57c62b4d135 service placement] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:34:14.968784 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bced74-fe72-415a-a204-f57c62b4d135 service placement] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:34:15.025886 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e7b70-7bf4-4b93-af63-ad53d04fd28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:34:15.026916 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e7b70-7bf4-4b93-af63-ad53d04fd28e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:34:15.027203 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e7b70-7bf4-4b93-af63-ad53d04fd28e admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:34:15.027507 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e7b70-7bf4-4b93-af63-ad53d04fd28e admin admin] PATH_INFO: `/v3/projects/3e8fe8f5ca4d410b8fcdad9e420bd49a` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:34:15.028638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09e7b70-7bf4-4b93-af63-ad53d04fd28e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e8fe8f5ca4d410b8fcdad9e420bd49a)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:34:15.030532 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09e7b70-7bf4-4b93-af63-ad53d04fd28e admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:34:15.146405 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-a09e7b70-7bf4-4b93-af63-ad53d04fd28e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e8fe8f5ca4d410b8fcdad9e420bd49a', 'request_id': 'req-a09e7b70-7bf4-4b93-af63-ad53d04fd28e'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:15.151917 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-a09e7b70-7bf4-4b93-af63-ad53d04fd28e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e8fe8f5ca4d410b8fcdad9e420bd49a', 'request_id': 'req-a09e7b70-7bf4-4b93-af63-ad53d04fd28e'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:15.214392 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-a09e7b70-7bf4-4b93-af63-ad53d04fd28e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e8fe8f5ca4d410b8fcdad9e420bd49a', 'request_id': 'req-a09e7b70-7bf4-4b93-af63-ad53d04fd28e'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:15.227146 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-a09e7b70-7bf4-4b93-af63-ad53d04fd28e admin admin] The token cache is being invalidate because project 3e8fe8f5ca4d410b8fcdad9e420bd49a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:34:15.227585 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-a09e7b70-7bf4-4b93-af63-ad53d04fd28e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:15.228231 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-a09e7b70-7bf4-4b93-af63-ad53d04fd28e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:15.229701 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1449/2867] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:34:14 2022] DELETE /identity/v3/projects/3e8fe8f5ca4d410b8fcdad9e420bd49a => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:34:15.233710 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf6665-6cbf-4352-b0ae-dd92e5b45f82 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:34:15.235146 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-f9bf6665-6cbf-4352-b0ae-dd92e5b45f82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:34:15.244899 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:34:15.248187 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf6665-6cbf-4352-b0ae-dd92e5b45f82 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:34:15.304306 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a063a-8841-4bee-b4c8-c73e34420ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:34:15.304937 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a063a-8841-4bee-b4c8-c73e34420ccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:34:15.305225 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a063a-8841-4bee-b4c8-c73e34420ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:34:15.305506 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a063a-8841-4bee-b4c8-c73e34420ccc admin admin] PATH_INFO: `/v3/users/ab1fc36506f44cb9bd55857cec1a8c16` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:34:15.306651 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a063a-8841-4bee-b4c8-c73e34420ccc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab1fc36506f44cb9bd55857cec1a8c16)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:34:15.308593 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a063a-8841-4bee-b4c8-c73e34420ccc admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:34:15.490702 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-436a063a-8841-4bee-b4c8-c73e34420ccc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab1fc36506f44cb9bd55857cec1a8c16'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:15.491330 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-436a063a-8841-4bee-b4c8-c73e34420ccc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab1fc36506f44cb9bd55857cec1a8c16'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:15.491916 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-436a063a-8841-4bee-b4c8-c73e34420ccc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab1fc36506f44cb9bd55857cec1a8c16'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:15.498853 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-436a063a-8841-4bee-b4c8-c73e34420ccc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab1fc36506f44cb9bd55857cec1a8c16'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:15.510830 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-436a063a-8841-4bee-b4c8-c73e34420ccc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab1fc36506f44cb9bd55857cec1a8c16'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:15.619356 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1419/2868] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:34:15 2022] DELETE /identity/v3/users/ab1fc36506f44cb9bd55857cec1a8c16 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:34:15.913391 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e7b70-7bf4-4b93-af63-ad53d04fd28e admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:34:15.914817 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-a09e7b70-7bf4-4b93-af63-ad53d04fd28e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:34:15.930187 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e7b70-7bf4-4b93-af63-ad53d04fd28e admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:34:15.947495 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:34:15.989197 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d12e64-9648-4357-9a04-7e65d101f2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:34:15.989915 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d12e64-9648-4357-9a04-7e65d101f2b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:34:15.990094 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d12e64-9648-4357-9a04-7e65d101f2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:34:15.990371 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d12e64-9648-4357-9a04-7e65d101f2b8 admin admin] PATH_INFO: `/v3/projects/10f6cbdea63f43c78e604a02940c6e7e` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:34:15.991523 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d12e64-9648-4357-9a04-7e65d101f2b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10f6cbdea63f43c78e604a02940c6e7e)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:34:15.993415 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d12e64-9648-4357-9a04-7e65d101f2b8 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:34:16.105896 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-e5d12e64-9648-4357-9a04-7e65d101f2b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10f6cbdea63f43c78e604a02940c6e7e', 'request_id': 'req-e5d12e64-9648-4357-9a04-7e65d101f2b8'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:16.112380 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-e5d12e64-9648-4357-9a04-7e65d101f2b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10f6cbdea63f43c78e604a02940c6e7e', 'request_id': 'req-e5d12e64-9648-4357-9a04-7e65d101f2b8'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:16.195372 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-e5d12e64-9648-4357-9a04-7e65d101f2b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10f6cbdea63f43c78e604a02940c6e7e', 'request_id': 'req-e5d12e64-9648-4357-9a04-7e65d101f2b8'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:16.207544 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-e5d12e64-9648-4357-9a04-7e65d101f2b8 admin admin] The token cache is being invalidate because project 10f6cbdea63f43c78e604a02940c6e7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:34:16.207831 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-e5d12e64-9648-4357-9a04-7e65d101f2b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:16.208768 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-e5d12e64-9648-4357-9a04-7e65d101f2b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:16.210268 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1450/2869] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:34:15 2022] DELETE /identity/v3/projects/10f6cbdea63f43c78e604a02940c6e7e => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:34:16.215473 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83ba72-0b49-4616-9609-5ab2e946dbfe None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:34:16.215762 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83ba72-0b49-4616-9609-5ab2e946dbfe None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:34:16.216038 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83ba72-0b49-4616-9609-5ab2e946dbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:34:16.236277 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-8e83ba72-0b49-4616-9609-5ab2e946dbfe None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:34:16.239723 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-8e83ba72-0b49-4616-9609-5ab2e946dbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:34:16.296532 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1420/2870] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:34:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:34:16.307251 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f41121-11f2-4bae-b80d-3bea8d3520e3 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:34:16.307794 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f41121-11f2-4bae-b80d-3bea8d3520e3 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:34:16.308306 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f41121-11f2-4bae-b80d-3bea8d3520e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:34:16.332885 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-66f41121-11f2-4bae-b80d-3bea8d3520e3 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:34:16.334117 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-66f41121-11f2-4bae-b80d-3bea8d3520e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:34:16.342530 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1451/2871] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:34:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:34:16.347060 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83ba72-0b49-4616-9609-5ab2e946dbfe None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:34:16.357200 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83ba72-0b49-4616-9609-5ab2e946dbfe None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:34:16.361868 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b0efa4-8a9e-4d11-973e-f9c86e731818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:34:16.362476 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b0efa4-8a9e-4d11-973e-f9c86e731818 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:34:16.362763 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b0efa4-8a9e-4d11-973e-f9c86e731818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:34:16.363036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b0efa4-8a9e-4d11-973e-f9c86e731818 admin admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:34:16.363723 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b0efa4-8a9e-4d11-973e-f9c86e731818 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:34:16.364036 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b0efa4-8a9e-4d11-973e-f9c86e731818 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:34:16.365656 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b0efa4-8a9e-4d11-973e-f9c86e731818 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:34:16.378462 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1421/2872] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:34:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:34:16.382846 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f41121-11f2-4bae-b80d-3bea8d3520e3 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:34:16.394381 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f41121-11f2-4bae-b80d-3bea8d3520e3 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:34:16.399132 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975f90c-69fa-48d8-9061-d96885f05197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:34:16.399764 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975f90c-69fa-48d8-9061-d96885f05197 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:34:16.400042 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975f90c-69fa-48d8-9061-d96885f05197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:34:16.400313 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975f90c-69fa-48d8-9061-d96885f05197 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:34:16.400975 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2975f90c-69fa-48d8-9061-d96885f05197 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:34:16.402867 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2975f90c-69fa-48d8-9061-d96885f05197 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:34:16.502728 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1452/2873] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:34:16 2022] POST /identity/v3/projects => generated 385 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:34:16.506964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b0efa4-8a9e-4d11-973e-f9c86e731818 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:34:16.518100 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b0efa4-8a9e-4d11-973e-f9c86e731818 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:34:16.525715 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:34:16.572954 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5569d649-970c-46bb-85dd-4583ede67e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:34:16.573553 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569d649-970c-46bb-85dd-4583ede67e91 admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:34:16.573833 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569d649-970c-46bb-85dd-4583ede67e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:34:16.574100 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569d649-970c-46bb-85dd-4583ede67e91 admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:34:16.574730 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5569d649-970c-46bb-85dd-4583ede67e91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:34:16.576635 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5569d649-970c-46bb-85dd-4583ede67e91 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:34:16.689426 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1422/2874] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:34:16 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:34:16.693593 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975f90c-69fa-48d8-9061-d96885f05197 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:34:16.704416 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975f90c-69fa-48d8-9061-d96885f05197 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:34:16.709129 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6492a93f-4629-49e1-b03b-c5051def2006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:34:16.709730 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492a93f-4629-49e1-b03b-c5051def2006 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:34:16.710066 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492a93f-4629-49e1-b03b-c5051def2006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:34:16.710344 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492a93f-4629-49e1-b03b-c5051def2006 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:34:16.710987 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6492a93f-4629-49e1-b03b-c5051def2006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:34:16.712876 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6492a93f-4629-49e1-b03b-c5051def2006 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:34:16.722882 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1453/2875] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:34:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 12:34:16.728692 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5569d649-970c-46bb-85dd-4583ede67e91 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:34:16.739281 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5569d649-970c-46bb-85dd-4583ede67e91 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:34:16.743978 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ab9cc-cce0-46dc-9821-09180a2702fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:34:16.744571 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ab9cc-cce0-46dc-9821-09180a2702fe admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:34:16.744854 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ab9cc-cce0-46dc-9821-09180a2702fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:34:16.745134 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ab9cc-cce0-46dc-9821-09180a2702fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:34:16.745807 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ab9cc-cce0-46dc-9821-09180a2702fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:34:16.747382 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ab9cc-cce0-46dc-9821-09180a2702fe admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:34:16.758590 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1423/2876] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:34:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 12:34:16.762810 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6492a93f-4629-49e1-b03b-c5051def2006 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:34:16.773276 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6492a93f-4629-49e1-b03b-c5051def2006 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:34:16.778307 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24edd4b5-9936-46d2-8f35-73ae2e5e56e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:34:16.778964 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24edd4b5-9936-46d2-8f35-73ae2e5e56e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:34:16.779242 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24edd4b5-9936-46d2-8f35-73ae2e5e56e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:34:16.779624 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24edd4b5-9936-46d2-8f35-73ae2e5e56e5 admin admin] PATH_INFO: `/v3/projects/cd4fee3e3e624e5b8b33bc7850e1168e/users/1629ed25594b4105b7aa9ab51434c1fb/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:34:16.794729 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24edd4b5-9936-46d2-8f35-73ae2e5e56e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd4fee3e3e624e5b8b33bc7850e1168e, user_id=1629ed25594b4105b7aa9ab51434c1fb, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:34:16.797920 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24edd4b5-9936-46d2-8f35-73ae2e5e56e5 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:34:16.895108 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1454/2877] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:34:16 2022] PUT /identity/v3/projects/cd4fee3e3e624e5b8b33bc7850e1168e/users/1629ed25594b4105b7aa9ab51434c1fb/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:34:16.905753 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe51a9-e61c-4240-978a-b7cb1dc549b6 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:34:16.906039 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe51a9-e61c-4240-978a-b7cb1dc549b6 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:34:16.906319 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe51a9-e61c-4240-978a-b7cb1dc549b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:34:16.938849 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-1cbe51a9-e61c-4240-978a-b7cb1dc549b6 None None] MFA Rules not processed for user `1629ed25594b4105b7aa9ab51434c1fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:34:16.940153 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-1cbe51a9-e61c-4240-978a-b7cb1dc549b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:34:17.003392 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1424/2878] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:34:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:34:17.014428 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24edd4b5-9936-46d2-8f35-73ae2e5e56e5 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:34:17.016751 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-24edd4b5-9936-46d2-8f35-73ae2e5e56e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:34:17.069121 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:34:17.074130 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24edd4b5-9936-46d2-8f35-73ae2e5e56e5 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:34:17.077893 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9283384-0a35-4f2e-95c8-e9eea371882f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:34:17.081174 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9283384-0a35-4f2e-95c8-e9eea371882f service cinder] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:34:17.081174 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9283384-0a35-4f2e-95c8-e9eea371882f service cinder] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:34:17.081174 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9283384-0a35-4f2e-95c8-e9eea371882f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:34:17.088566 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9283384-0a35-4f2e-95c8-e9eea371882f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:34:17.089996 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9283384-0a35-4f2e-95c8-e9eea371882f service cinder] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:34:17.099509 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1455/2879] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:34:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:34:17.118413 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe51a9-e61c-4240-978a-b7cb1dc549b6 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:34:17.182627 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe51a9-e61c-4240-978a-b7cb1dc549b6 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:34:17.187371 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d773c608-a00e-438a-b9fb-1d43f024d8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:34:17.188213 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773c608-a00e-438a-b9fb-1d43f024d8b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:34:17.188500 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773c608-a00e-438a-b9fb-1d43f024d8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:34:17.188778 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773c608-a00e-438a-b9fb-1d43f024d8b6 admin admin] PATH_INFO: `/v3/users/1629ed25594b4105b7aa9ab51434c1fb` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:34:17.189930 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d773c608-a00e-438a-b9fb-1d43f024d8b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1629ed25594b4105b7aa9ab51434c1fb)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:34:17.192607 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d773c608-a00e-438a-b9fb-1d43f024d8b6 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:34:17.220568 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:34:17.382433 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-d773c608-a00e-438a-b9fb-1d43f024d8b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1629ed25594b4105b7aa9ab51434c1fb'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:17.383107 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-d773c608-a00e-438a-b9fb-1d43f024d8b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1629ed25594b4105b7aa9ab51434c1fb'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:17.383762 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-d773c608-a00e-438a-b9fb-1d43f024d8b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1629ed25594b4105b7aa9ab51434c1fb'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:17.391831 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-d773c608-a00e-438a-b9fb-1d43f024d8b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1629ed25594b4105b7aa9ab51434c1fb'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:17.403691 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-d773c608-a00e-438a-b9fb-1d43f024d8b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1629ed25594b4105b7aa9ab51434c1fb'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:17.488594 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1425/2880] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:34:17 2022] DELETE /identity/v3/users/1629ed25594b4105b7aa9ab51434c1fb => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:34:17.493481 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f8f42-e95a-41d2-bcbc-d83f872dc523 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:34:17.493776 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f8f42-e95a-41d2-bcbc-d83f872dc523 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:34:17.494060 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f8f42-e95a-41d2-bcbc-d83f872dc523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:34:17.513007 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-5e0f8f42-e95a-41d2-bcbc-d83f872dc523 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:34:17.514220 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-5e0f8f42-e95a-41d2-bcbc-d83f872dc523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:34:17.574983 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1456/2881] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:34:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:34:17.584894 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d773c608-a00e-438a-b9fb-1d43f024d8b6 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:34:17.586561 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-d773c608-a00e-438a-b9fb-1d43f024d8b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:34:17.643227 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d773c608-a00e-438a-b9fb-1d43f024d8b6 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:34:17.647032 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d35f76-e1a5-4a85-9099-46fbfe457d75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:34:17.647674 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d35f76-e1a5-4a85-9099-46fbfe457d75 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:34:17.647958 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d35f76-e1a5-4a85-9099-46fbfe457d75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:34:17.648229 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d35f76-e1a5-4a85-9099-46fbfe457d75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:34:17.657221 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d35f76-e1a5-4a85-9099-46fbfe457d75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:34:17.658648 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d35f76-e1a5-4a85-9099-46fbfe457d75 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:34:17.667974 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1426/2882] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:34:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:34:18.077779 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0f8f42-e95a-41d2-bcbc-d83f872dc523 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:34:18.079331 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-5e0f8f42-e95a-41d2-bcbc-d83f872dc523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:34:18.097654 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0f8f42-e95a-41d2-bcbc-d83f872dc523 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:34:18.099742 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5352abc3-3c9c-47c6-8ad6-55cd2ff8ed38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:34:18.100377 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352abc3-3c9c-47c6-8ad6-55cd2ff8ed38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:34:18.100658 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352abc3-3c9c-47c6-8ad6-55cd2ff8ed38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:34:18.101062 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352abc3-3c9c-47c6-8ad6-55cd2ff8ed38 admin admin] PATH_INFO: `/v3/projects/cd4fee3e3e624e5b8b33bc7850e1168e` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:34:18.102311 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5352abc3-3c9c-47c6-8ad6-55cd2ff8ed38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd4fee3e3e624e5b8b33bc7850e1168e)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:34:18.104569 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5352abc3-3c9c-47c6-8ad6-55cd2ff8ed38 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:34:18.226147 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-5352abc3-3c9c-47c6-8ad6-55cd2ff8ed38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd4fee3e3e624e5b8b33bc7850e1168e', 'request_id': 'req-5352abc3-3c9c-47c6-8ad6-55cd2ff8ed38'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:18.232120 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-5352abc3-3c9c-47c6-8ad6-55cd2ff8ed38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd4fee3e3e624e5b8b33bc7850e1168e', 'request_id': 'req-5352abc3-3c9c-47c6-8ad6-55cd2ff8ed38'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:18.304685 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-5352abc3-3c9c-47c6-8ad6-55cd2ff8ed38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd4fee3e3e624e5b8b33bc7850e1168e', 'request_id': 'req-5352abc3-3c9c-47c6-8ad6-55cd2ff8ed38'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:18.318007 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-5352abc3-3c9c-47c6-8ad6-55cd2ff8ed38 admin admin] The token cache is being invalidate because project cd4fee3e3e624e5b8b33bc7850e1168e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:34:18.318297 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-5352abc3-3c9c-47c6-8ad6-55cd2ff8ed38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:18.318955 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-5352abc3-3c9c-47c6-8ad6-55cd2ff8ed38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:18.320537 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1457/2883] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:34:18 2022] DELETE /identity/v3/projects/cd4fee3e3e624e5b8b33bc7850e1168e => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:34:18.325415 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315f38f-0903-4f27-b5bb-10c547caad08 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:34:18.325810 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315f38f-0903-4f27-b5bb-10c547caad08 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:34:18.325999 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315f38f-0903-4f27-b5bb-10c547caad08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:34:18.351393 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-b315f38f-0903-4f27-b5bb-10c547caad08 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:34:18.352652 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-b315f38f-0903-4f27-b5bb-10c547caad08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:34:18.412412 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1427/2884] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:34:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:34:18.420148 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2edcdf-a895-47e2-84e2-5be808470cf1 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:34:18.420440 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2edcdf-a895-47e2-84e2-5be808470cf1 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:34:18.420722 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2edcdf-a895-47e2-84e2-5be808470cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:34:18.441817 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-2e2edcdf-a895-47e2-84e2-5be808470cf1 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:34:18.443035 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-2e2edcdf-a895-47e2-84e2-5be808470cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:34:18.450910 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1458/2885] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:34:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:34:18.455082 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b315f38f-0903-4f27-b5bb-10c547caad08 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:34:18.465371 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b315f38f-0903-4f27-b5bb-10c547caad08 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:34:18.469964 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c163d263-138d-4cc2-b6d2-de0e935fc117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:34:18.470566 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c163d263-138d-4cc2-b6d2-de0e935fc117 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:34:18.470851 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c163d263-138d-4cc2-b6d2-de0e935fc117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:34:18.471127 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c163d263-138d-4cc2-b6d2-de0e935fc117 admin admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:34:18.471805 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c163d263-138d-4cc2-b6d2-de0e935fc117 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 18 12:34:18.472116 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c163d263-138d-4cc2-b6d2-de0e935fc117 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:34:18.473702 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c163d263-138d-4cc2-b6d2-de0e935fc117 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:34:18.485704 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1428/2886] 10.222.0.98 () {58 vars in 1153 bytes} [Thu Aug 18 12:34:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 18 12:34:18.490558 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2edcdf-a895-47e2-84e2-5be808470cf1 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:34:18.501773 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2edcdf-a895-47e2-84e2-5be808470cf1 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:34:18.506641 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7b833-c53e-4c87-852c-7696bf6382d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:34:18.507240 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7b833-c53e-4c87-852c-7696bf6382d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:34:18.507616 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7b833-c53e-4c87-852c-7696bf6382d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:34:18.507898 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7b833-c53e-4c87-852c-7696bf6382d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:34:18.508518 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a7b833-c53e-4c87-852c-7696bf6382d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:34:18.510425 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a7b833-c53e-4c87-852c-7696bf6382d0 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:34:18.592038 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1459/2887] 10.222.0.98 () {60 vars in 1153 bytes} [Thu Aug 18 12:34:18 2022] POST /identity/v3/projects => generated 391 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:34:18.596866 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c163d263-138d-4cc2-b6d2-de0e935fc117 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:34:18.610139 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c163d263-138d-4cc2-b6d2-de0e935fc117 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:34:18.641752 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:34:18.672339 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c8159-3e0a-45d8-9cf8-ad1d65c65c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:34:18.673062 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c8159-3e0a-45d8-9cf8-ad1d65c65c5e admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:34:18.673360 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c8159-3e0a-45d8-9cf8-ad1d65c65c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:34:18.673636 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c8159-3e0a-45d8-9cf8-ad1d65c65c5e admin admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:34:18.674379 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c8159-3e0a-45d8-9cf8-ad1d65c65c5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:34:18.676368 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c8159-3e0a-45d8-9cf8-ad1d65c65c5e admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:34:18.797221 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1429/2888] 10.222.0.98 () {60 vars in 1144 bytes} [Thu Aug 18 12:34:18 2022] POST /identity/v3/users => generated 296 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 18 12:34:18.801481 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7b833-c53e-4c87-852c-7696bf6382d0 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:34:18.813956 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7b833-c53e-4c87-852c-7696bf6382d0 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:34:18.818537 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895666c6-26f1-423b-b130-41f056d1dcb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:34:18.824334 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895666c6-26f1-423b-b130-41f056d1dcb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:34:18.824334 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895666c6-26f1-423b-b130-41f056d1dcb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:34:18.824334 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895666c6-26f1-423b-b130-41f056d1dcb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:34:18.824334 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895666c6-26f1-423b-b130-41f056d1dcb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:34:18.824334 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895666c6-26f1-423b-b130-41f056d1dcb1 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:34:18.832769 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1460/2889] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 18 12:34:18.836912 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c8159-3e0a-45d8-9cf8-ad1d65c65c5e admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:34:18.847699 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c8159-3e0a-45d8-9cf8-ad1d65c65c5e admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:34:18.852326 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900290ca-6c6c-44ca-bf50-5e7287a6b129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:34:18.852924 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900290ca-6c6c-44ca-bf50-5e7287a6b129 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:34:18.853207 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900290ca-6c6c-44ca-bf50-5e7287a6b129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:34:18.853486 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900290ca-6c6c-44ca-bf50-5e7287a6b129 admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:34:18.854128 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900290ca-6c6c-44ca-bf50-5e7287a6b129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:34:18.855739 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900290ca-6c6c-44ca-bf50-5e7287a6b129 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:34:18.866512 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1430/2890] 10.222.0.98 () {58 vars in 1122 bytes} [Thu Aug 18 12:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 18 12:34:18.870643 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895666c6-26f1-423b-b130-41f056d1dcb1 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:34:18.881256 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895666c6-26f1-423b-b130-41f056d1dcb1 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:34:18.885844 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f2ae60-b062-448a-a609-7ddd509d94c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:34:18.886538 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2ae60-b062-448a-a609-7ddd509d94c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:34:18.893229 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2ae60-b062-448a-a609-7ddd509d94c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:34:18.893229 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2ae60-b062-448a-a609-7ddd509d94c2 admin admin] PATH_INFO: `/v3/projects/0e9659488f114ebd8691559eb239d589/users/9871ca2c3bf148b8a8faff940ba3e782/roles/c81e28908b9b49a38d33978df4d11716` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:34:18.904268 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f2ae60-b062-448a-a609-7ddd509d94c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e9659488f114ebd8691559eb239d589, user_id=9871ca2c3bf148b8a8faff940ba3e782, role_id=c81e28908b9b49a38d33978df4d11716)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:34:18.907464 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f2ae60-b062-448a-a609-7ddd509d94c2 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:34:18.996427 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1461/2891] 10.222.0.98 () {60 vars in 1483 bytes} [Thu Aug 18 12:34:18 2022] PUT /identity/v3/projects/0e9659488f114ebd8691559eb239d589/users/9871ca2c3bf148b8a8faff940ba3e782/roles/c81e28908b9b49a38d33978df4d11716 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:34:19.008631 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219b2d1-575d-489d-a6f1-c7f616b313da None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:34:19.008918 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219b2d1-575d-489d-a6f1-c7f616b313da None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:34:19.009222 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219b2d1-575d-489d-a6f1-c7f616b313da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:34:19.037343 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-1219b2d1-575d-489d-a6f1-c7f616b313da None None] MFA Rules not processed for user `9871ca2c3bf148b8a8faff940ba3e782`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:34:19.038554 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-1219b2d1-575d-489d-a6f1-c7f616b313da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:34:19.100535 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1431/2892] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:34:19 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 18 12:34:19.111111 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f2ae60-b062-448a-a609-7ddd509d94c2 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:34:19.112870 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-21f2ae60-b062-448a-a609-7ddd509d94c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:34:19.124113 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:34:19.170596 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f2ae60-b062-448a-a609-7ddd509d94c2 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:34:19.174325 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e37b35b-5a64-45e1-8add-00fba2a87b9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c41b6fb3cb064a49b9236077a8faed54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:34:19.174924 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37b35b-5a64-45e1-8add-00fba2a87b9c service cinder] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:34:19.175202 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37b35b-5a64-45e1-8add-00fba2a87b9c service cinder] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:34:19.175508 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37b35b-5a64-45e1-8add-00fba2a87b9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:34:19.183794 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e37b35b-5a64-45e1-8add-00fba2a87b9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:34:19.185300 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e37b35b-5a64-45e1-8add-00fba2a87b9c service cinder] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:34:19.194954 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1462/2893] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:34:19 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:34:29.097678 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:34:29.220573 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=22 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:34:39.957697 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1219b2d1-575d-489d-a6f1-c7f616b313da None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:34:40.020117 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1219b2d1-575d-489d-a6f1-c7f616b313da None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:34:40.024866 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0d911-9205-46b9-b0f8-b5afd487ea3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:34:40.025499 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0d911-9205-46b9-b0f8-b5afd487ea3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:34:40.025881 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0d911-9205-46b9-b0f8-b5afd487ea3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:34:40.026665 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0d911-9205-46b9-b0f8-b5afd487ea3a admin admin] PATH_INFO: `/v3/users/9871ca2c3bf148b8a8faff940ba3e782` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:34:40.027843 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf0d911-9205-46b9-b0f8-b5afd487ea3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9871ca2c3bf148b8a8faff940ba3e782)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:34:40.029788 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf0d911-9205-46b9-b0f8-b5afd487ea3a admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:34:40.193851 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-1cf0d911-9205-46b9-b0f8-b5afd487ea3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9871ca2c3bf148b8a8faff940ba3e782'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:40.194501 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-1cf0d911-9205-46b9-b0f8-b5afd487ea3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9871ca2c3bf148b8a8faff940ba3e782'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:40.195191 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-1cf0d911-9205-46b9-b0f8-b5afd487ea3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9871ca2c3bf148b8a8faff940ba3e782'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:40.202935 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-1cf0d911-9205-46b9-b0f8-b5afd487ea3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9871ca2c3bf148b8a8faff940ba3e782'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:40.215311 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-1cf0d911-9205-46b9-b0f8-b5afd487ea3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9871ca2c3bf148b8a8faff940ba3e782'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:40.277178 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1432/2894] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:34:39 2022] DELETE /identity/v3/users/9871ca2c3bf148b8a8faff940ba3e782 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:34:40.282219 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72b80a-587a-4179-b60d-3698a7063120 None None] REQUEST_METHOD: `POST` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:34:40.282797 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72b80a-587a-4179-b60d-3698a7063120 None None] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:34:40.282797 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72b80a-587a-4179-b60d-3698a7063120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:34:40.303975 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.auth.core [None req-af72b80a-587a-4179-b60d-3698a7063120 None None] MFA Rules not processed for user `6c92bf2b004241b3b54d40ae3aaaefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81863) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 18 12:34:40.305254 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-af72b80a-587a-4179-b60d-3698a7063120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:34:40.364202 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1463/2895] 10.222.0.98 () {58 vars in 958 bytes} [Thu Aug 18 12:34:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 18 12:34:40.374595 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0d911-9205-46b9-b0f8-b5afd487ea3a admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:34:40.376196 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-1cf0d911-9205-46b9-b0f8-b5afd487ea3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:34:40.433383 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0d911-9205-46b9-b0f8-b5afd487ea3a admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:34:40.437024 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d4427c-2868-4992-b376-258d6c8a12b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:34:40.437624 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4427c-2868-4992-b376-258d6c8a12b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:34:40.437907 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4427c-2868-4992-b376-258d6c8a12b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:34:40.438181 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4427c-2868-4992-b376-258d6c8a12b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:34:40.448483 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d4427c-2868-4992-b376-258d6c8a12b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:34:40.449925 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d4427c-2868-4992-b376-258d6c8a12b3 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:34:40.460469 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1433/2896] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:34:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:34:40.815787 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af72b80a-587a-4179-b60d-3698a7063120 None None] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:34:40.817200 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-af72b80a-587a-4179-b60d-3698a7063120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:34:40.830791 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af72b80a-587a-4179-b60d-3698a7063120 None None] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:34:40.835552 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4dc5e-ac56-4027-8ac6-87c01cd36349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:34:40.836190 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4dc5e-ac56-4027-8ac6-87c01cd36349 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:34:40.836473 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4dc5e-ac56-4027-8ac6-87c01cd36349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:34:40.836747 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4dc5e-ac56-4027-8ac6-87c01cd36349 admin admin] PATH_INFO: `/v3/projects/0e9659488f114ebd8691559eb239d589` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:34:40.838908 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea4dc5e-ac56-4027-8ac6-87c01cd36349 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e9659488f114ebd8691559eb239d589)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:34:40.841347 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea4dc5e-ac56-4027-8ac6-87c01cd36349 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:34:40.997408 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-2ea4dc5e-ac56-4027-8ac6-87c01cd36349 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e9659488f114ebd8691559eb239d589', 'request_id': 'req-2ea4dc5e-ac56-4027-8ac6-87c01cd36349'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:41.002992 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-2ea4dc5e-ac56-4027-8ac6-87c01cd36349 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e9659488f114ebd8691559eb239d589', 'request_id': 'req-2ea4dc5e-ac56-4027-8ac6-87c01cd36349'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:41.070644 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-2ea4dc5e-ac56-4027-8ac6-87c01cd36349 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e9659488f114ebd8691559eb239d589', 'request_id': 'req-2ea4dc5e-ac56-4027-8ac6-87c01cd36349'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:41.082592 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-2ea4dc5e-ac56-4027-8ac6-87c01cd36349 admin admin] The token cache is being invalidate because project 0e9659488f114ebd8691559eb239d589 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81863) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:34:41.082885 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-2ea4dc5e-ac56-4027-8ac6-87c01cd36349 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:41.083563 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-2ea4dc5e-ac56-4027-8ac6-87c01cd36349 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:34:41.085042 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1464/2897] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:34:40 2022] DELETE /identity/v3/projects/0e9659488f114ebd8691559eb239d589 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:34:50.455408 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:34:51.079701 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:35:51.904303 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d4427c-2868-4992-b376-258d6c8a12b3 service neutron] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:35:51.906527 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-e1d4427c-2868-4992-b376-258d6c8a12b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:35:51.920005 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d4427c-2868-4992-b376-258d6c8a12b3 service neutron] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:35:51.970325 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce77c5-c762-4caf-8950-27f74c11e32d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:35:51.971035 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce77c5-c762-4caf-8950-27f74c11e32d service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:35:51.971444 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce77c5-c762-4caf-8950-27f74c11e32d service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:35:51.971729 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce77c5-c762-4caf-8950-27f74c11e32d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:35:51.973520 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-52ce77c5-c762-4caf-8950-27f74c11e32d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:35:52.054557 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ce77c5-c762-4caf-8950-27f74c11e32d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:35:52.055938 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ce77c5-c762-4caf-8950-27f74c11e32d service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:35:52.067476 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1434/2898] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:35:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:35:53.253371 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4dc5e-ac56-4027-8ac6-87c01cd36349 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:35:53.254844 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-2ea4dc5e-ac56-4027-8ac6-87c01cd36349 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:35:53.267638 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4dc5e-ac56-4027-8ac6-87c01cd36349 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:35:53.271258 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a582e-5fa5-422c-86eb-e8b499846a9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:35:53.271893 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a582e-5fa5-422c-86eb-e8b499846a9d service neutron] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:35:53.272179 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a582e-5fa5-422c-86eb-e8b499846a9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:35:53.272457 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a582e-5fa5-422c-86eb-e8b499846a9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:35:53.274724 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-de5a582e-5fa5-422c-86eb-e8b499846a9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:35:53.288278 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5a582e-5fa5-422c-86eb-e8b499846a9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:35:53.290396 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5a582e-5fa5-422c-86eb-e8b499846a9d service neutron] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:35:53.301330 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1465/2899] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:35:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 18 12:36:02.063055 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=54 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:36:03.296483 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=6 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:36:06.359576 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce77c5-c762-4caf-8950-27f74c11e32d service neutron] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:36:06.361126 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-52ce77c5-c762-4caf-8950-27f74c11e32d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:36:06.374326 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce77c5-c762-4caf-8950-27f74c11e32d service neutron] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:36:06.378333 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d25a8-d0ee-4fd8-86fd-6391e832b3ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '185644affb284c2b91e604cf99803ffb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:36:06.378965 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d25a8-d0ee-4fd8-86fd-6391e832b3ec service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:36:06.379249 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d25a8-d0ee-4fd8-86fd-6391e832b3ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:36:06.379569 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d25a8-d0ee-4fd8-86fd-6391e832b3ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:36:06.381305 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-c43d25a8-d0ee-4fd8-86fd-6391e832b3ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:36:06.451562 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43d25a8-d0ee-4fd8-86fd-6391e832b3ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:36:06.453019 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43d25a8-d0ee-4fd8-86fd-6391e832b3ec service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:36:06.462770 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1435/2900] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:36:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:36:13.917334 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a582e-5fa5-422c-86eb-e8b499846a9d service neutron] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:36:13.919521 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-de5a582e-5fa5-422c-86eb-e8b499846a9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:36:13.935688 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a582e-5fa5-422c-86eb-e8b499846a9d service neutron] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:36:13.940567 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2bc1ca-6700-4932-b0d2-0a732f7541c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:36:13.941206 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2bc1ca-6700-4932-b0d2-0a732f7541c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:36:13.941512 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2bc1ca-6700-4932-b0d2-0a732f7541c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:36:13.941791 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2bc1ca-6700-4932-b0d2-0a732f7541c2 admin admin] PATH_INFO: `/v3/users/3907830851364a73b4d3715280a132bc` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:36:13.942950 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2bc1ca-6700-4932-b0d2-0a732f7541c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3907830851364a73b4d3715280a132bc)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:36:13.944901 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2bc1ca-6700-4932-b0d2-0a732f7541c2 admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:36:14.096452 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-fe2bc1ca-6700-4932-b0d2-0a732f7541c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3907830851364a73b4d3715280a132bc'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:36:14.103750 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-fe2bc1ca-6700-4932-b0d2-0a732f7541c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3907830851364a73b4d3715280a132bc'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:36:14.104447 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-fe2bc1ca-6700-4932-b0d2-0a732f7541c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3907830851364a73b4d3715280a132bc'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:36:14.116542 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-fe2bc1ca-6700-4932-b0d2-0a732f7541c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3907830851364a73b4d3715280a132bc'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:36:14.162073 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-fe2bc1ca-6700-4932-b0d2-0a732f7541c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3907830851364a73b4d3715280a132bc'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:36:14.168115 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1466/2901] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:36:13 2022] DELETE /identity/v3/users/3907830851364a73b4d3715280a132bc => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:36:14.387403 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d25a8-d0ee-4fd8-86fd-6391e832b3ec service neutron] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:36:14.398236 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d25a8-d0ee-4fd8-86fd-6391e832b3ec service neutron] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:36:14.456496 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24618a56-f36d-4073-a281-64e58b469636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:36:14.457123 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24618a56-f36d-4073-a281-64e58b469636 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:36:14.457404 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24618a56-f36d-4073-a281-64e58b469636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:36:14.457679 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24618a56-f36d-4073-a281-64e58b469636 admin admin] PATH_INFO: `/v3/projects/c1534d6a880c4f44b741ff5b208663bc` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:36:14.458788 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24618a56-f36d-4073-a281-64e58b469636 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1534d6a880c4f44b741ff5b208663bc)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:36:14.460703 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24618a56-f36d-4073-a281-64e58b469636 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:36:14.538551 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-24618a56-f36d-4073-a281-64e58b469636 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1534d6a880c4f44b741ff5b208663bc', 'request_id': 'req-24618a56-f36d-4073-a281-64e58b469636'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:36:14.544561 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-24618a56-f36d-4073-a281-64e58b469636 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1534d6a880c4f44b741ff5b208663bc', 'request_id': 'req-24618a56-f36d-4073-a281-64e58b469636'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:36:14.585297 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-24618a56-f36d-4073-a281-64e58b469636 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1534d6a880c4f44b741ff5b208663bc', 'request_id': 'req-24618a56-f36d-4073-a281-64e58b469636'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:36:14.597251 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-24618a56-f36d-4073-a281-64e58b469636 admin admin] The token cache is being invalidate because project c1534d6a880c4f44b741ff5b208663bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:36:14.597541 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-24618a56-f36d-4073-a281-64e58b469636 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:36:14.598211 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-24618a56-f36d-4073-a281-64e58b469636 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:36:14.599714 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1436/2902] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:36:14 2022] DELETE /identity/v3/projects/c1534d6a880c4f44b741ff5b208663bc => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:36:14.603881 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2bc1ca-6700-4932-b0d2-0a732f7541c2 admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:36:14.605831 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-fe2bc1ca-6700-4932-b0d2-0a732f7541c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:36:14.618982 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2bc1ca-6700-4932-b0d2-0a732f7541c2 admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:36:14.675288 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bcba6e-da87-4104-ab50-1c4fc46d37ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:36:14.675878 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcba6e-da87-4104-ab50-1c4fc46d37ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:36:14.676164 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcba6e-da87-4104-ab50-1c4fc46d37ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:36:14.676443 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcba6e-da87-4104-ab50-1c4fc46d37ae admin admin] PATH_INFO: `/v3/users/9fdeb664774a4c6e88ad61d3488960c0` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:36:14.677639 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bcba6e-da87-4104-ab50-1c4fc46d37ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fdeb664774a4c6e88ad61d3488960c0)` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:36:14.679596 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bcba6e-da87-4104-ab50-1c4fc46d37ae admin admin] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:36:14.851096 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-45bcba6e-da87-4104-ab50-1c4fc46d37ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fdeb664774a4c6e88ad61d3488960c0'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:36:14.858667 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-45bcba6e-da87-4104-ab50-1c4fc46d37ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fdeb664774a4c6e88ad61d3488960c0'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:36:14.859346 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-45bcba6e-da87-4104-ab50-1c4fc46d37ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fdeb664774a4c6e88ad61d3488960c0'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:36:14.871967 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-45bcba6e-da87-4104-ab50-1c4fc46d37ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fdeb664774a4c6e88ad61d3488960c0'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:36:14.873642 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:36:14.940022 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.notifications [None req-45bcba6e-da87-4104-ab50-1c4fc46d37ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fdeb664774a4c6e88ad61d3488960c0'} {{(pid=81863) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:36:14.946569 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1467/2903] 10.222.0.98 () {58 vars in 1224 bytes} [Thu Aug 18 12:36:14 2022] DELETE /identity/v3/users/9fdeb664774a4c6e88ad61d3488960c0 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:36:15.210189 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24618a56-f36d-4073-a281-64e58b469636 admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:36:15.213838 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-24618a56-f36d-4073-a281-64e58b469636 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:36:15.226609 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24618a56-f36d-4073-a281-64e58b469636 admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:36:15.256999 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:36:15.300851 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f9193-c21f-4076-90c3-077abdd9d1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c92bf2b004241b3b54d40ae3aaaefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c192a2eba4e24841893715c8478b559c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:36:15.302626 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f9193-c21f-4076-90c3-077abdd9d1b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:36:15.302626 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f9193-c21f-4076-90c3-077abdd9d1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:36:15.302626 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f9193-c21f-4076-90c3-077abdd9d1b4 admin admin] PATH_INFO: `/v3/projects/4a0434ddfbda4102a6217e5af1b07d17` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:36:15.303254 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400f9193-c21f-4076-90c3-077abdd9d1b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a0434ddfbda4102a6217e5af1b07d17)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:36:15.305227 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400f9193-c21f-4076-90c3-077abdd9d1b4 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:36:15.419816 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-400f9193-c21f-4076-90c3-077abdd9d1b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a0434ddfbda4102a6217e5af1b07d17', 'request_id': 'req-400f9193-c21f-4076-90c3-077abdd9d1b4'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:36:15.425606 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-400f9193-c21f-4076-90c3-077abdd9d1b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a0434ddfbda4102a6217e5af1b07d17', 'request_id': 'req-400f9193-c21f-4076-90c3-077abdd9d1b4'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:36:15.493724 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-400f9193-c21f-4076-90c3-077abdd9d1b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a0434ddfbda4102a6217e5af1b07d17', 'request_id': 'req-400f9193-c21f-4076-90c3-077abdd9d1b4'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:36:15.505450 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-400f9193-c21f-4076-90c3-077abdd9d1b4 admin admin] The token cache is being invalidate because project 4a0434ddfbda4102a6217e5af1b07d17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81862) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 18 12:36:15.505750 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-400f9193-c21f-4076-90c3-077abdd9d1b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:36:15.506397 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-400f9193-c21f-4076-90c3-077abdd9d1b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 18 12:36:15.507973 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1437/2904] 10.222.0.98 () {58 vars in 1233 bytes} [Thu Aug 18 12:36:15 2022] DELETE /identity/v3/projects/4a0434ddfbda4102a6217e5af1b07d17 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 18 12:36:25.031029 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:36:25.502261 dv-cinder-836027-7-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 18 12:38:48.788607 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bcba6e-da87-4104-ab50-1c4fc46d37ae admin admin] Authenticating user token {{(pid=81863) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 18 12:38:48.790024 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-45bcba6e-da87-4104-ab50-1c4fc46d37ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:38:48.932504 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bcba6e-da87-4104-ab50-1c4fc46d37ae admin admin] Validating token access rules against request {{(pid=81863) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 18 12:38:48.992773 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62530320-beb6-4e70-af10-9272ef0c1e90 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193637acadc14ec4a48ab79566ba3dab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f676a5169464d83af4267db25025728', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81863) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 18 12:38:48.993393 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62530320-beb6-4e70-af10-9272ef0c1e90 service placement] REQUEST_METHOD: `GET` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 18 12:38:48.993679 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62530320-beb6-4e70-af10-9272ef0c1e90 service placement] SCRIPT_NAME: `/identity` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 18 12:38:48.993959 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62530320-beb6-4e70-af10-9272ef0c1e90 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81863) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 18 12:38:48.995799 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.fernet_utils [None req-62530320-beb6-4e70-af10-9272ef0c1e90 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81863) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 18 12:38:49.008058 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62530320-beb6-4e70-af10-9272ef0c1e90 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 18 12:38:49.014925 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62530320-beb6-4e70-af10-9272ef0c1e90 service placement] RBAC: Authorization granted {{(pid=81863) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 18 12:38:49.019798 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: [pid: 81863|app: 0|req: 1468/2905] 10.222.0.98 () {60 vars in 1365 bytes} [Thu Aug 18 12:38:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 18 12:38:59.015715 dv-cinder-836027-7-iscsi devstack@keystone.service[81863]: DEBUG dbcounter [-] [81863] Writing DB stats keystone:SELECT=89 {{(pid=81863) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}